Undergraduate dentistry student’s perception on objective structured practical examination (OSPE): A cross-sectional study

Introduction

Objective structured practical examination (OSPE) is a globally implemented and beneficial system that is utilized for assessing students in practical examination. It was derived from the term objective structured clinical examination (OSCE) which originated in 1975 and was later modified by Harden and Gleeson in 1979.1, 2 In the conventional practical examination (CPE), the examiner cannot observe all the students’ continuously while performing the practical, which mostly extends over a long period. Students are evaluated only by the final result of the experiment, and the marks awarded vary markedly from one examiner to the other. In OSPE, the practical is performed in the presence of the examiner, who evaluates the psychomotor skills of the students with reference to a checklist; it is therefore based entirely on the students’ performance with little scope for subjectivity.3, 4

OSPE is a better tool of assessment as it can assess all three domains –Cognitive (analytical questions in unobserved stations), psychomotor (step-wise demonstrations of procedures) and affective (Communication skills).5 Hence this study was planned with the aim to assess students’ perception on objective structured practical examination (OSPE) – as a tool for the formative assessment of practical skills of undergraduate students in dental material. The objectives are

Materials and Methods

This cross- sectional study was carried out, at our institute, after obtaining the Institutional Ethics Committee approval. Study is based on the analysis of the results of Second Internal Assessment practical examination given to the entire second BDS batch of 32 students in Dental Material, in the Department of Conservative Dentistry and Endodontics at Government Dental College and Hospital. The study did not affect the regular examination schedule of the Department. During the Second Internal assessment examination in Dental Material, the students were assessed for their practical using both conventional practical examination (CPE) and objective structured practical examination (OSPE). For convenience and feasibility, the practical examination was conducted over 2 days with a gap of one week after CPE.

CPE (Figure 1) was conducted as per Second BDS Curriculum. CPE for Dental Material practical examination includes - Spotting- Identification of materials, Manipulation of Zinc Phosphate cement and Silver Amalgam and Viva voce. In CPE two dental materials were kept for identification, manipulation of zinc phosphate and silver amalgam was observed and examined by two teachers and viva voce was conducted by two teachers.

Figure 1

Steps for conventional practical examination (CPE) and objective structured practical examination (OSPE)

The students were assessed as follows, at five OSPE stations.

Station 1- Question station (QS) -Identification of two dental Material

Station 2- Procedure Station (PS) - Manipulation of Zinc Phosphate cement,

Station 3- Procedure Station (PS) - Manipulation of Silver Amalgam,

Station 4 and 5- Response Station (RS).

Four minutes were given at each QS, PS and Response Station. Each station was designed along with the checklist, by the authors. The stations were selected to represent the learning objectives from the Dental Material curriculum. For the procedure station, each point on the check list was scored according to the binary system, that is, ‘Yes/No’ scale, by the observer, and marks were given accordingly. For the Dental Material examination, faculty from Conservative Dentistry and Endodontics acted as the observer at the procedure station along with the check list- one faculty per station. Students required 50% marks to pass both type of tests. After the examination, feedback was obtained from the students with the help of a pre‑validated questionnaire, in the first theory class that followed. Questions pertaining to the students’ perception regarding OSPE compared to CPE, the difficulties faced and their opinions regarding inclusion of OSPE as an assessment method in dental material, were included. On the basis of OSPE evaluation, weakness of each student was evaluated, noted and discussed with students.

Observations and Result

The data was analyzed using SPSS V16.0 software. The paired t‑test was used to compare the marks obtained in conventional practical examination (CPE) and Objective structured practical examination (OSPE). As the OSPE was held on two consecutive days, with different exercises, the marks obtained by the students were also compared using the paired t‑test. The value of P < 0.05 was considered statistically insignificant. The performance of students by OSPE was better, mean scores were 14.89 SD (1.68) & 15.75 SD (2.06) for CPE & OSPE respectively. The difference in marks was insignificant when paired t test was applied, when we compared result of all the students.

Discussion

In our Institution, assessment of practical skills for formative and summative examinations are done by traditional methods. Many colleges and Universities have started using objective structured practical examination OSPE / objective structured clinical examination OSCE for assessment in formative as well as summative exams.5 So this study was undertaken to study the feasibility of using this type of assessment in subject of Dental Material.

In this study, assessment of the students was done by both OSPE and CPE. Statistical insignificant difference (Table 1) in the average scores obtained by the students was present, giving the impression that both methods are comparable in efficacy. This finding was similar to the findings of Malhotra et al. 20133 and contrary to the findings of Hasan et al., 2009.6 The higher scores in OSPE are most likely due to the unambiguous instructions given to the students and also due to the unambiguous criteria of evaluation. These findings differ from the previous studies, which show a significant difference between the CPE and OSPE scores.7, 8 Results of our study also showed that the CPE and OSPE scores differed. 81% students found OSPE easier and 100% student responded positively for OSPE. 84% students perceived OSPE (Figure 2, Figure 3) fairer and objective than the conventional examination in accordance with NF Desai (2014),9 Malhotra SD (2013)3 and Natu NV (1994)7 and Roy V (2004).8

Figure 2

Figure 3

Graph showing objectivity of OSPE n=32

(84% students found OSPE unbiased exam.13% found it biased & 3% students were uncertain, when objectivity was checked)

Figure 4

Response to OSPE for improvement in knowledge, attitude and skill

(69% students agree and 28% strongly agree that OSPE improves KAP, while 3% disagree for KAP)

Communication is one of the most important skills in management of the patients by the physicians. A station meant to evaluate the communication domain was included, in the form of giving verbal specific answers to specific questions - such as the surface area used on glass slab for manipulation of Zinc Phosphate cement.

OSPE was conducted with five stations, which could affect the reliability of the test. Station 1- Question Station - Cognitive (analytical questions in unobserved stations), Station 2 and 3-Procedure Station- Psychomotor skill based (step-wise demonstrations of procedures), Station 4 and 5- Response Station- Cognitive, Psychomotor and Attitude (Shows completed procedure and explain). The strength of the study was that all the students were exposed to both the types of examinations, CPE and OSPE, and were in position to give their opinion. 97% of students perceived improvement in Knowledge, skill and attitude (Figure 4) which is in agreement with previous studies.

Feedback given by the teachers (Table 2 identification of weakness) to the students after the exam helped them to correct their mistakes and they became aware about what was expected from them in that particular experiment and OSPE has helped them to learn procedures in a systematic manner. Therefore, OSPE can also be used as a teaching/learning tool.3

Conclusion

OSPE is feasible and acceptable to the students for the internal assessment of practical skills in undergraduate training in Dental Education. It has been well accepted and appreciated by the students and therefore could be recommended as a tool for teaching and assessment.

Source of Funding

None.

Conflict of Interest

The authors declare no conflict of interest.