|Year : 2019 | Volume
| Issue : 3 | Page : 144-149
Learning environment assessment on the pediatric residency program at Dr. Moewardi Hospital, Surakarta, Indonesia
Annang Giri Moelyo, Muhammad Riza, Fadhilah Tia Nur, Hari Wahyu Nugroho, Sri Martuti, Dwi Hidayah, Endang Dewi Lestari, Harsono Salimo
Department of Pediatric, Dr. Moewardi Hospital, Universitas Sebelas Maret, Surakarta, Indonesia
|Date of Web Publication||14-Jan-2020|
Annang Giri Moelyo
Department of Pedicatric, Dr. Moewardi Hospital, Universitas Sebelas Maret, Jalan Kolonel Soetarto 132, Surakarta 57126
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: The learning environment plays an important role in resident education, and the Postgraduate Hospital Educational Environment Measure (PHEEM) questionnaire is a tool to assess the student's perception of the learning environment. The aim of this study was to assess the perceptions of the learning environment in 2012 and 2017 on the pediatric residency program at Dr. Moewardi Hospital, Universitas Sebelas Maret, Surakarta, Indonesia.
Methods: A modified PHEEM questionnaire was distributed in April 2012 and September 2017 to all ongoing pediatric residents. The learning environments (sex, grade, perceptions of autonomy, teaching, and social support) in 2012 and 2017 were analyzed by Mann–Whitney test and Kruskal–Wallis test.
Results: There were 32 and 49 participants in 2012 and 2017, respectively. The overall PHEEM score (mean ± standard deviation) was 105.3 ± 19.1 in 2012 and 116.4 ± 12.1 in 2017. The mean scores for perceptions of autonomy, teaching, and social support were 36.3 ± 6.6, 41.0 ± 7.7, and 27.9 ± 5.7, respectively, in 2012, and 41.0 ± 4.7, 44.7 ± 4.4, and 30.7 ± 3.9, respectively, in 2017. In 2012, 21%, 27%, and 18% of items on perceptions of autonomy, teaching, and social support, respectively, rated >3; in 2017, these values were 35%, 53%, and 36%.
Conclusion: The perceptions of the learning environment for the pediatric residency program at Dr. Moewardi Hospital Surakarta in 2012 and 2017 were rated more positively, but there remains room for improvement. In 2017, there were more perceived difficulties with autonomy and social support.
Keywords: Learning environment, pediatric residency, postgraduate hospital educational environment measure
|How to cite this article:|
Moelyo AG, Riza M, Nur FT, Nugroho HW, Martuti S, Hidayah D, Lestari ED, Salimo H. Learning environment assessment on the pediatric residency program at Dr. Moewardi Hospital, Surakarta, Indonesia. J Nat Sc Biol Med 2019;10, Suppl S1:144-9
|How to cite this URL:|
Moelyo AG, Riza M, Nur FT, Nugroho HW, Martuti S, Hidayah D, Lestari ED, Salimo H. Learning environment assessment on the pediatric residency program at Dr. Moewardi Hospital, Surakarta, Indonesia. J Nat Sc Biol Med [serial online] 2019 [cited 2020 Jan 22];10, Suppl S1:144-9. Available from: http://www.jnsbm.org/text.asp?2019/10/3/144/275573
| Introduction|| |
A hospital is also the most important place for clinical training. Pediatric residents, as part of specialty programs, predominantly use their learning time in a hospital setting, where they must integrate their patient care responsibilities, socioeconomic problems, learning duties, onsite overtime duties, and collaboration with other residents and teaching staff. In this complex learning setting, the environment plays an important role. An appropriate learning environment will encourage a motivation to study and work well in the hospital,, whereas a poor learning environment becomes a stressor for residents that might lead to burnout., Methods have been developed for the assessment of the learning environment in a medical setting. The Postgraduate Hospital Educational Environment Measurement (PHEEM) is an assessment tool for postgraduate students., The PHEEM questionnaire assesses the overall learning environment and perceptions of autonomy, teaching, and social support., The Faculty of Medicine, Universitas Sebelas Maret, Surakarta, Indonesia, has had a pediatric residency program since 2002. Dr. Moewardi Hospital is the main teaching hospital for the program. Regular assessment of the learning environment was needed to understand environmental changes and identify any persistent and high-priority problems. We aimed to assess the learning environment for the pediatric residency program at Moewardi Hospital, Universitas Sebelas Maret, Surakarta, Indonesia, in 2012 and 2017.
| Methods|| |
This analytic observational study was conducted in April 2012 and September 2017. Two versions of a modified PHEEM questionnaire were distributed to all ongoing pediatric residents of Universitas Sebelas Maret during the study. The study received ethical approval from the Health Research Ethics Committee, Dr. Moewardi General Hospital, School of Medicine, Sebelas Maret University, approval number 216/II/HREC. We obtained written consent from participants before participation in the study. The first version (distributed in 2012) comprised 40 questions in English based on an Australian PHEEM form, with a Bahasa Indonesia explanation of each question. The explanations were adjusted for the setting of Dr. Moewardi Hospital. The second version (distributed in 2017) included 40 questions in Bahasa Indonesia based on an Australian PHEEM form. The questionnaire had been validated for interns. The responses use a 5-point Likert scale (0–4; 0 strongly disagree, 1 disagree, 2 uncertain, 3 agree, and 4 strongly agree), except for questions 7, 8, 11, and 13, which were scored inversely because these are negative statements., Cronbach's α was used as the test of reliability and a score of >0.8 was regarded as indicating reliability. The total PHEEM score was classified as follows: 0–40 very poor, 41–80 many problems, 81–120 more positive than negative but room for improvement, and 121–160 excellent. Perception of autonomy was defined from 14 questions (1, 4, 5, 8, 9, 11, 14, 17, 18, 29, 30, 32, 34, and 40), perception of teaching from 15 questions (2, 3, 6, 10, 12, 15, 21, 22, 23, 27, 28, 31, 33, 37, and 39), and perception of social support from 11 questions (7, 13, 16, 19, 20, 24, 25, 26, 35, 36, and 38). The response rate was calculated based on the number of fully completed questionnaires. The interpretations were classified as follows: perception of autonomy, 0–14 very poor, 15–28 a negative view of one's role, 29–42 a more positive perception of one's role, and 43–56 an excellent perception of one's role; perception of teaching, 0–15 very poor quality, 16–30 in need of some retraining, 31–45 moving in the right direction, and 46–60 model teachers; perception of social support, 0–11 nonexistent, 12–22 not a pleasant place, 23–33 more pros than cons, and 34–44 a good supportive environment.
The data were analyzed using SPSS version 22 software (IBM Corp, Armonk, NY, New York, USA). The results for the PHEEM scores (total and each item) were expressed as means and standard deviations. The results for learning environment (by sex and grade) in 2012 and 2017 were analyzed by Mann–Whitney test and Kruskal–Wallis test. P < 0.05 was considered statistically significant. The proportions of items relating perceptions of autonomy, teaching, and social support that were rated >3 were expressed as percentages.
| Results|| |
This study assessed the data from 2012 and 2017. Thirty-two of 37 and 49 of 55 residents in those years were enrolled in this study, giving response rates of 86.5% and 89%, respectively. The male-to-female ratios in 2012 and 2017 were 1.0 and 0.63, respectively. The proportions of participants by grade (junior, intermediate, and senior) were 16/9/7 and 15/12/22. Reliability testing showed Cronbach's α values of 0.94 and 0.93, respectively [Table 1].
The total PHEEM score in both years fell within the classification “more positive than negative (learning environment) but room for improvement” (105.3 ± 19.1 and 116.4 ± 12.1). The results for male and female participants and for junior-, intermediate-, and senior-grade participants were also within the same classification [Table 2], although the results for 2017 approached the upper end of this range [Table 3].
|Table 2: Postgraduate Hospital Educational Environment Measure scores for 2012 and 2017|
Click here to view
|Table 3: Perception of autonomy, teaching, and social support in 2012 and 2017|
Click here to view
The questions and the responses of pediatric residents for each item of the PHEEM questionnaire are shown in [Table 4]. Items 7, 8, 11, and 13 were scored inversely because they included negative statements, and in the questionnaire, a higher score indicates a more positive response. There were two items in the social support section that scored <2 in 2012 (20 and 26) and one in 2017 (26). The other items all scored >2. The lowest scores for perceptions of autonomy in both years were for items 9 (there is an informative junior doctor's handbook) and 14 (there are clear clinical protocols in this post). A new problem with autonomy emerged in 2017, i.e., item 32 (my workload in this job is fine). The lowest scores for perceptions of teaching in both years were for item 21 (there is access to an educational program relevant to my needs). New problems with teaching in 2017 were items 3 (I have protected educational time in this post) and 31 (my clinical teachers are accessible). The lowest perceptions about social support in both years were similar, i. e., items 20 (this hospital has good-quality accommodation for junior doctors, especially when on call), 25 (there is a no-blame culture in this post), and 26 (there are adequate catering facilities when I am on call).
|Table 4: Responses of pediatric residents to items of the Postgraduate Hospital Educational Environment Measure questionnaire in 2012 and 2017|
Click here to view
In 2012, three of 14 items in autonomy (5, 11, and 29), four of 15 items in teaching (10, 12, 28, and 37), and two of 11 items in social support (16 and 35) in year 2012 were rated >3 (agree). In 2017, five of 14 items in autonomy (1, 5, 11, 34, and 40), eight of 15 items in teaching (2, 6, 10, 12, 15, 23, 28, and 37), and four of 11 items in social support (7, 13, 16, and 35) were scored >3 [Table 4].
| Discussion|| |
Self-assessment in pediatric residency program of Universitas Sebelas Maret Surakarta was periodically performed every 5 years. Self-assessment comprised a strength weakness opportunity thread (SWOT) analysis and a learning environment assessment and then was used to arrange strategic planning for the next 5 years. During 5 years (from 2012 to 2017), there were many priority programs related to teaching methods, the use of Workplace based assessment and also hospital and institutional accreditations that might change perceptions of learning environment. This study assessed the perceptions of pediatric residents about their learning environment in 2012 and 2017. The response rates of our study were quite high (86.5% and 89%); other studies have had response rates of 20%–100%.,,, The reliability of this study was more than 0.8, which is consistent with that of other studies.,,,, The questionnaire had high internal consistency. In our program, there was a change in the male-to-female ratio between 2012 and 2017, in that the proportion of female residents in the pediatric program increased. Although many studies have reported no conclusive differences between men and women, this trend must be investigated further in the context of our patriarchal society., Both assessments resulted in scores within the same classification (more positive than negative but room for improvement). Some studies have reported lower scores for pediatric residents, e.g., Saudi Arabia (82.64), Ireland (82.88), and Pakistan (88.16).,, Multicenter studies in Saudi Arabia reported a total score of 100. A study by Wall et al. compiled data from Brazil, Chile, Netherlands, and the UK and produced a total score of 105. A South African study assessing pediatric interns reported similar findings. Almost all studies reported scores within the same classification. Residents may encounter some difficulties and, in some cases, suffer burnout during their residency. Previous studies have reported the prevalence of difficulties in residency as 3%–10%, and the learning environment has been correlated with burnout in residency programs., From the perspective of program directors, these difficulties were related to the workplace and the incidence was higher in university teaching hospitals than in regional hospitals. The prevalence of burnout among pediatric residents itself has been shown to vary (15%–70%)., Although the present study was not designed to assess difficulties and burnout in our residency program, junior-grade residents reported more stress and heavier workloads than more senior residents. At this level, they also faced problems in adapting to the work/learning environment. In the view of our teaching staff, the junior grade is the toughest stage of the residency. The results for 2012 showed that junior- and intermediate-grade residents reported the lowest scores, although the difference between grades was not significant. However, in 2017 the scores for all grades were almost identical. We cannot explain these differences. They may be the result of different proportions of participants in each grade in 2012 and 2017. The duration of residency was also not a significant influence in multicenter studies in Pakistan and Saudi Arabia., As in our study, men and women did not differ in their perceptions of the learning environment., Another study reported that anxiety and depression were common during the 1st year of residency, and that female students had greater symptoms of anxiety than men. Further research is needed to assess specifically the workload, the prevalence of and risk factors for anxiety, and burnout in pediatric residency, including factors such as duration of residency and sex.
The perceptions of autonomy, teaching, and social support in our study were positive, which is consistent with those reported in other studies., One study in Saudi Arabia found positive responses for autonomy and teaching but negative ones for social support. Our participants gave ratings of <2 only for perception of social support. In 2012, the items that scored low were this hospital has good quality accommodation for junior doctors, especially when on call and there are adequate catering facilities when I am on call. In 2017, catering facilities were still a problem, but not accommodation. Another study in Saudi Arabia reported that the availability of an informative junior doctor's handbook, workload, a blame culture, and adequate catering facilities when on call all scored <2. Other studies of interns and residents not specific for pediatrics reported that inadequate catering services, performing inappropriate tasks, being paged inappropriately, having suitable access to career advisers, an informative handbook, an informative induction program, and senior staff who effectively utilized learning opportunities were scored <2.,,, The factor common to our study was the problem with the catering facilities that might have influenced the learning environment. This important issue could prompt the teaching hospital to improve the facilities.
The perception of program directors was that the most common problems in teaching were related to feedback, which differed from the perception of the residents in this study. Specifically, the persistent problems reported in both years were the lack of an informative junior doctor's handbook, clear clinical protocols, access to an educational program, good-quality accommodation for junior doctors, a no-blame culture, and adequate catering facilities. The new problems that emerged in 2017 were related to autonomy and teaching: workload, protected educational time, and accessibility of teachers. Residents' workload in our academic hospital might related to increasing number and complexity of pediatric cases, lots of administrative tasks due to hospital accreditation, and no officially limitation of working hours for residents. The increased workload had impact on educational time. Accessibility of teachers' problems in 2017 might be caused by increased workload (quantity and complexity) of patients' care in hospital, administrative tasks for medical staff, and some medical staff took an advance education for their subspecialties. Evaluation of these items needs to be performed regularly.
Overall, 35%, 53%, and 36% of items in perceptions of autonomy, teaching, and social support, respectively, were rated >3. This differs from the results of a multicenter study in Saudi Arabia in which the higher scores predominantly related to autonomy. Compared with the data for 2012, in 2017, 3/14 (21%), 4/15 (27%), and 2/11 (18%) of items relating to perception of autonomy, teaching, and social support, respectively, were rated >3. Therefore, the present study shows that, in 2017, our residents faced more difficulties in autonomy and social support than in teaching. Residency program directors should evaluate and develop approaches to change the residents' perceptions of these two aspects.
Our study had some limitations. First, the methods for gathering information from participants differed in 2012 and 2017. In 2012, we used an English-based questionnaire with an explanation in Bahasa, whereas in 2017, a Bahasa translation of an English-based questionnaire was used. Clearly, there is an issue related to internal validity, although Cronbach's α indicated high internal consistency. Hence, we did not compare the observations for the 2 years. However, this study was useful to assess residents' perceptions about their learning environment in 2012 and 2017. There were some changes during that period, not only the participants' characteristics (sex proportion, number of participants, social economic background, and learning environment before), but also the external change (mini and meso-curriculum, the hospital rule, and properties). We suggest the use of a Bahasa Indonesia version of PHEEM for further research to allow valid comparison of different periods and to assess each item specifically. Second, the participants in 2012 and 2017 were not the same individuals. In 2017, the proportion of women was larger and the number of senior-grade participants was greater. Although the results did not differ for men and women, this change in proportion might have introduced a bias in perceptions. The participants' psychological background and other risk factors (age, marital status, family, financial problems, or the origin of university during medical student) need to be explored to understand their perceptions. A cohort study is needed to analyze the changes in the learning environment, the association between the perception of the learning environment and the residents' motivation and burnout, and whether the perception of the learning environment has any correlation with academic achievement. In conclusion, the perceptions of the learning environment of the pediatric residency program at Dr. Moewardi Hospital Surakarta in 2012 and 2017 were rated more positive than negative, but there is room for improvement. By 2017, there were more difficulties with perceptions of autonomy and social support than of teaching. New emerging problems were the workload, educational time, and accessibility of teachers.
| Conclusion|| |
The perceptions of the learning environment of the pediatric residency program at Dr. Moewardi Hospital Surakarta in 2012 and 2017 were rated more positive than negative but there is room for improvement. By 2017, there were more difficulties with perceptions of autonomy and social support than of teaching. New emerged problems were the workload, educational time and accessibility of teachers.
We would like to thank Zhafira and Beta Adelia Septiana for assisting with the research.
Financial support and sponsorship
The 3rd ICE on IMERI Committee supported the peer review and manuscript preparation of this article.
Conflicts of interest
There are no conflicts of interest.
| References|| |
Flaherty GT, Connolly R, O'Brien T. Measurement of the postgraduate educational environment of junior doctors training in medicine at an Irish University teaching hospital. Ir J Med Sci 2016;185:565-71.
Raharjanti NW, Ramadianto AS. Validity and reliability of postgraduate hospital educational environment measure – Bahasa Indonesia version. J Pendidik Kedokt Indones 2016;5:157-62.
Jamjoom RS, Park YS. Assessment of pediatric residents burnout in a tertiary academic centre. Saudi Med J 2018;39:296-300.
van Vendeloo SN, Prins DJ, Verheyen CC, Prins JT, van den Heijkant F, van der Heijden FM, et al.
The learning environment and resident burnout: A national study. Perspect Med Educ 2018;7:120-5.
Roff S, McAleer S, Skinner A. Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK. Med Teach 2005;27:326-31.
Miles S, Swift L, Leinster SJ. The Dundee ready education environment measure (DREEM): A review of its adoption and use. Med Teach 2012;34:e620-34.
Wall D, Clapham M, Riquelme A, Vieira J, Cartmill R, Aspegren K, et al.
Is PHEEM a multi-dimensional instrument? An international perspective. Med Teach 2009;31:e521-7.
Lucas MN, Samarage DK. Trainees' perception of the clinical learning environment in the postgraduate training programmes in pediatrics. Sri Lanka J Child Health 2008;37:76-80.
Vieira JE. The postgraduate hospital educational environment measure (PHEEM) questionnaire identifies quality of instruction as a key factor predicting academic achievement. Clinics (Sao Paulo) 2008;63:741-6.
Gooneratne IK, Munasinghe SR, Siriwardena C, Olupeliyawa AM, Karunathilake I. Assessment of psychometric properties of a modified PHEEM questionnaire. Ann Acad Med Singapore 2008;37:993-7.
Al-Marshad S, Alotaibi G. evaluation of clinical educational environment at king Fahad hospital of Dammam University using the postgraduate hospital education environment measure (PHEEM) inventory. Educ Med J. 2011;3:e6-14.
Peterlini M, Tibério IF, Saadeh A, Pereira JC, Martins MA. Anxiety and depression in the first year of medical residency training. Med Educ 2002;36:66-72.
Bari A, Khan RA, Rathore AW. Postgraduate residents' perception of the clinical learning environment; use of postgraduate hospital educational environment measure (PHEEM) in Pakistani context. J Pak Med Assoc 2018;68:417-22.
BuAli WH, Khan AS, Al-Qahtani MH, Aldossary S. Evaluation of hospital-learning environment for pediatric residency in Eastern Region of Saudi Arabia. J Educ Eval Health Prof 2015;12:14.
Naidoo KL, Van Wyk JM, Adhikari M. The learning environment of paediatric interns in South Africa. BMC Med Educ 2017;17:235.
Christensen MK, O'Neill L, Hansen DH, Norberg K, Mortensen LS, Charles P. Residents in difficulty: A mixed methods study on the prevalence, characteristics, and sociocultural challenges from the perspective of residency program directors. BMC Med Educ 2016;16:69.
McClafferty H, Brooks AJ, Chen MK, Brenner M, Brown M, Esparham A, et al.
Pediatric integrative medicine in residency program: Relationship between lifestyle behaviors and burnout and wellbeing measures in first-year residents. Children(Basel) 2018;5:54.
Algaidi SA. Assessment of educational environment for interns using postgraduate hospital educational environment measure (PHEEM). J Taibah Univ Med Sci 2010;5:1-2.
[Table 1], [Table 2], [Table 3], [Table 4]