|Year : 2013 | Volume
| Issue : 1 | Page : 103-107
Objective structured practical examination in biochemistry: An experience in Medical College, Kolkata
Dipankar Kundu1, HN Das2, Gargi Sen2, Manish Osta3, T Mandal1, Divyendu Gautam4
1 Department of Biochemistry, Medical College, Kolkata, India
2 Department of Biochemistry, R.G. Kar Medical College, Kolkata, India
3 Department of Pathology, Medical College, Kolkata, India
4 Department of Surgery, Medical College, Kolkata, India
|Date of Web Publication||20-Feb-2013|
Medical College, College Street, Kolkata 73, West Bengal
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: Undergraduate medical examination is undergoing extensive re evaluation with new core educational objectives being defined. Consequently, new exam systems have also been designed to test the objectives. Objective structured practical examination (OSPE) is one of them. Objectives: To introduce OSPE as a method of assessment of practical skills and learning and to determine student satisfaction regarding the OSPE. Furthermore, to explore the faculty perception of OSPE as a learning and assessment tool. Materials and Methods: The first M.B.B.S students of 2011 12 batch of Medical College, Kolkata, were the subjects for the study. OSPE was organized and conducted on "Identification of Unknown Abnormal Constituents in Urine." Coefficient of reliability of questions administered was done by calculating Cronbach's alpha. A questionnaire on various components of the OSPE was administered to get the feedback. Results: 16 students failed to achieve an average of 50% or above in the assessment. However, 49 students on an average achieved >75%, 52 students achieved between 65% and 75%, and 29 students scored between 50% and 65%. Cronbach's alpha of the questions administered showed to be having high internal consistency with a score of 0.80. Ninety nine percent of students believed that OSPE helps them to improve and 81% felt that this type of assessment fits in as both learning and evaluation tools. Faculty feedback reflected that such assessment tested objectivity, measured practical skills better, and eliminated examiner bias to a greater extent. Conclusion: OSPE tests different desired components of competence better and eliminated examiner bias. Student feedback reflects that such assessment helps them to improve as it is effective both as teaching and evaluation tools.
Keywords: Biochemistry, evaluation, objective structured practical examination
|How to cite this article:|
Kundu D, Das H N, Sen G, Osta M, Mandal T, Gautam D. Objective structured practical examination in biochemistry: An experience in Medical College, Kolkata. J Nat Sc Biol Med 2013;4:103-7
|How to cite this URL:|
Kundu D, Das H N, Sen G, Osta M, Mandal T, Gautam D. Objective structured practical examination in biochemistry: An experience in Medical College, Kolkata. J Nat Sc Biol Med [serial online] 2013 [cited 2020 Jan 29];4:103-7. Available from: http://www.jnsbm.org/text.asp?2013/4/1/103/107268
| Introduction|| |
It is a well-known fact that assessment drives learning. A single examination does not fulfill all the functions of assessment, such as assessing knowledge, comprehension, skills, motivation, and feedback.  Written examinations (essays and multiple choices) test cognitive knowledge, which is only one aspect of the competency. Structuring of questions and assessment through highlighting on objectivity has been emphasized and gained importance in the practical evaluation. The objective structured practical examination (OSPE) is now an accepted tool in the assessment of practical skills in both Pre- and Para-clinical subjects. However, there are no strict or limiting guidelines on the types of scenario that are used in the OSPE examinations. In the UK, the USA, Canada, and indeed most reputable colleges of medicine, the OSPE is the standard mode of assessment of competency, clinical skills, and counseling sessions satisfactorily complementing cognitive knowledge testing in essay writing and objective examination.  Several universities adopted a similar pattern of practical evaluation, which is un-uniform and largely subjective. Biochemistry departments in many medical schools have been using the OSPE as an assessment method for assessing students' performance in laboratory exercises.
Examiner variability significantly affects scoring. The marks awarded generally reflect only the global performance of the candidate and are not based on demonstration of individual competencies. In the university examination, there used to be frequent complaints from external examiners insisting that the existing examination pattern is tedious and time-consuming. The OSPE is a versatile multipurpose evaluative tool that can be utilized to evaluate students in practical assessment. It assesses competency, based on objective testing through direct observation. It is comprised of several "stations" in which examinees are expected to perform a variety of practical tasks within a specified time period against criteria formulated to the practical skill, thus demonstrating competency of skills and/or attitudes. OSPE has been used to evaluate those areas most critical to perform by students, such as the ability to obtain/interpret data, solve the problem, teach, and communicate. Any attempt to evaluate these critical areas in the old-fashioned practical examination will seem to be assessing theory rather than simulating practical performance.  An earlier innovation in this regard is the objective structured clinical examination (OSCE) later extended to the OSPE described in 1975 and in greater detail in 1979 by Harden and his group.  These methods with some modifications have stood the test of time and have largely overcome the problems of the conventional clinical/practical examinations mentioned earlier. In view of this, we tried the system of OSPE for the assessment of practical in the subject of biochemistry for the first time. Students usually learn only the material on which they are assessed. They do not go beyond the learning issues. If a test requires memorization of facts, they are driven to do that. This will lead to a situation where they adopt a surface approach to learning. Research shows that the type of assessment method adopted can influence student learning.  If the assessment pattern consists of a variety of methods that demand understanding of the subject matter, this problem can be solved to some extent. To test the earlier observation that a single exam does not fulfill all the functions of assessment, such as assessing knowledge, comprehension and skills, motivation, and providing feedback, we developed an evaluation system.
We undertook this study to evaluate whether OSPE could be a method of learning and assessment of practical skills in biochemistry and to explore the student, to determine student satisfaction regarding the OSPE as a method of assessment of laboratory exercises and to explore the faculty perception of OSPE as a learning and assessment tool.
| Materials and Methods|| |
The first M.B.B.S students admitted for 2011-2012 batch of Medical College were the subjects for the study. After successfully completing the syllabus pertaining to the topic on "Identification of Unknown Abnormal Constituents in Urine" in practical and "Unknown Abnormal Constituents in Urine" in theory, OSPE notification was announced 30 days in advance. A single examination does not fulfill all the functions of assessment. This study was undertaken to determine the reliability and student satisfaction regarding the OSPE as a method of assessment of laboratory exercises in biochemistry before implementing it in the forthcoming university examination. Before administering this tool for evaluation, all the staff members involved in designing and conducting OSPE were trained by attending an "Workshop on OSPE/OSCE" conducted by Medical Education Unit, Medical College. Ready-made and peer agreed upon check list formed the basis of assessment in procedure station. Structured questions were formed for question stations and key answers for the same were also prepared. Since the assessment was being carried out for the first time, the students were oriented toward such a system in advance before administering the tool. A total of 150 students were assessed. The assessment was conducted for a period of 6 days. Each day assessment was limited to 25 students only. Each student was assessed by attending four procedures and eight question stations. Each station was designed such that the task could be completed comfortably within 5 min. Coefficient of reliability of questions administered was done by calculating Cronbach's alpha.  A questionnaire on various components of the OSPE was administered to get the feedback.
| Results|| |
Among the 150 students, 146 students were present and took this OSPE exercise. Sixteen students failed to achieve an average of 50% or above in the assessment. However, 49 students on an average achieved > 75%, 52 students achieved between 65% and 75%, and 29 students scored between 50% and 65%. This has been shown in [Figure 1].
The mean scores of each station and the score obtained by calculation of Cronbach's alpha for testing the internal consistency of the questions administered are depicted in [Table 1]. Student feedback analysis of response to various aspects of OSPE is depicted in [Table 2]. Feedback from faculty on OSPE as an evaluation system has been shown in [Table 3]. Feedback on most appreciated aspect about OSPE is depicted in [Figure 2].
|Figure 2: Feedback on most appreciated aspects about objective structured practical examination by students|
Click here to view
|Table 1: The mean scores of each station and the score obtained by calculation of cronbach's alpha for testing the internal consistency of the questions administered|
Click here to view
|Table 2: Student feedback analysis of response to various aspects of objective structured practical examination|
Click here to view
|Table 3: Feedback from faculty on objective structured practical examination as an evaluation system|
Click here to view
| Discussion|| |
Over the years, increasing experience with the procedure has led to the use of OSPE not merely as an evaluation tool but as a teaching method. This has largely been attributable to the feedback that OSPE gives both to students and teachers. Among 150 students in 1 st M.B.B.S (2011-12) batch, four students could not attend due to their personal reasons. Of the 146 students, 101 students performance was highly satisfactory, who scored >65% of marks on an average. However, 16 students did not manage to get even 50% of average marks as their performance was equally poor in both performance and question stations. Questions administered were checked for coefficient of reliability by calculating Cronbach's alpha.  It showed that the questions administered were highly consistent with a score of 0.80. Evaluated marks of question station and check list of procedure station were made available to the students, who appreciated what they achieved and identified and where they need to improve. Feedback given by students was constructive and showed high acceptance, which are presented in [Table 2]. Feroze and his team have also reported to have got an appreciable feedback.  Majority of students appreciated orientation toward OSPE, syllabus and relevance of questions asked. Many students found that the manner in which the assessment was conducted was comfortable. Ninety-nine percent of students believed that OSPE helps them to improve and 81% felt that this type of assessment fits in as both learning and evaluation tools. However, 65% of students expressed that OSPE to be introduced partially in the final exams. A vast majority of students enjoyed OSPE because of its objectivity, more student assessment in less time, and uniformity. Majority of faculty felt that such exercises need to be given more frequently. However, 60% of faculty agreed upon the use of OSPE in both formative and summative evaluations.
The university examination is conducted in both forenoon and afternoon sessions for 6 consecutive days. In the university examination, external examiners used to complain about the extensiveness of the examination. Students used to complain about the irrelevant questions asked by the examiners and also the subjectivity of the examination. They used to complain that the questions asked in the performance exercises varied in difficulty, giving rise to much variation in the scores. Here, OSPE was adopted with the intention of restricting the examination only in the forenoon sessions, thereby reducing the total time, to make the assessment uniform for all students, and also to reduce the stress of students by making them go through only one round of examination instead of two rounds. The OSPE covers a broad range of skills much wider than a conventional examination. The scoring is objective, since standards of competence are pre-set and agreed check lists are used for scoring. Examiner variability can be reduced by adopting structured practical examination. In addition to the above points, OSPE ensures integration of teaching and evaluation. Variety maintains student's interest. There is increased faculty-student interaction. OSPE is adaptable to local needs. A large number of students can be tested within a short time.
From the students' point of view, the OSPE was acceptable and generated wide appreciation. Feedback from the students indicated that students were in favor of the OSPE. The feedback provided scope for improvement. This study reveals the importance of the role of students in developing new assessment tool. The study provided scope for refining the method. This type of assessment serves as a tool for testing multiple dimensions of student performance because it tests both skills as in performance exercises and knowledge as in OSPE.  The majority of students had the opinion that multiple choice questions were less stressful. Our findings correlate with the earlier findings that multiple choice questions test more of factual recall.  Nevertheless, if framed properly, multiple choice questions could be used to assess different levels of intellectuality. Our results agree with the earlier findings that a single type of assessment alone does not meet all the criteria for evaluating student performance. It also helps teachers to think about innovative methods of teaching and evaluation to improve the relevance of biochemistry and to modify question format to improve relevance and comprehension of questions in the succeeding exams.
| Conclusion|| |
In conclusion, OSPE has several distinct advantages. From our first experience, we found that OSPE was more objective, measured practical skills better, and eliminated examiner bias. Student feedback reflects that such assessment helps them to improve as it is effective both as teaching and evaluation tools. Faculty participated in organizing OSPE felt that such exercises can be given frequently for formative evaluation before introducing it in summative evaluation. We have outlined the features of the evaluation system followed in our setup, and based on the feedback, we consider that it would help students to develop different learning skills and make them better learners. Experience and experimentation will inevitably result in the refinement of the OSPE as a tool for learning and evaluation. However, in the current situation, it may not be realistic to expect its inclusion in the formal summative evaluation schedule of universities. However, it is feasible in view of the tremendous advantages that it offers, to include the formative (day to day) assessment of students to improve their clinical competence and to derive an objective score for internal assessment.
| Acknowledgments|| |
We would like to thank Ms. Mohua Bhattacharya, technical staff for her active participation in our study. Special thanks to the management of Medical College, Kolkata.
| References|| |
|1.||Abraham RR, Raghavendra R, Surekha K, Asha K. A trial of the objective structured practical examination in physiology at Melaka Manipal Medical College, India. Adv Physiol Educ 2009;33:21-3. |
|2.||Abraham RR, Upadhya S, Torke S, Ramnarayan K. Student perspectives of assessment by TEMM model in physiology. Adv Physiol Educ 2005;29:94-7. |
|3.||Ananthakrishnan N. Objective structured clinical/practical examination (OSCE/OSPE). J Postgrad Med 1993;39:82-4. |
|4.||Harden RM, Gleeson FA. Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ 1979;13:41-54. |
|5.||Harden RM, Stevenson M, Downie WW, Wilson GM. Assessment of clinical competence using objective structured examination. Br J Med Educ 1975;1:447-51. |
|6.||Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16:297-334. |
|7.||Bland JM, Altman DG. Statistics notes: Cronbach's alpha. BMJ 1997;314:372. |
|8.||Feroze M, Jacob AJ. OSPE in pathology. Indian J Pathol Microbiol 2002;45:53-7. |
|9.||Stiliman PL, Brown DR, Redfield DL, Sabors DL. Construct validation of the Arizona clinical interview rating scale. Educ Psychol Meas 1977;37:1031-8. |
|10.||Rahman N, Ferdousi S, Hoq N, Amin R, Kabir J. Evaluation of objective structured practical examination and traditional practical examination. Mymensingh Med J 2007;16:7-11. |
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3]