INTRODUCTION
Acquiring hands-on surgical skills remains a challenge for surgical residents, as various factors limit their opportunities to achieve competency. These include reduced operative exposure, an increase in non-clinical administrative duties, work hour restrictions, evolving patient expectations, and financial pressures on consultants to increase productivity.1–3 As a result, up to 85% of graduating residents are deemed inept to perform some surgeries independently.4–6 An assessment of general surgery residents also revealed that only 40.3% of them were near-independent in performing core surgical procedures.4,5 Moreover, surgical techniques are ever-evolving, requiring a steep learning curve with extensive repetition to attain proficiency. These factors will continue to challenge residency programs in developing skilled and independent surgical consultants.
Current training methods in orthopaedic surgery rely on a combination of technique guides, instructional videos, cadaveric dissections, the use of SawBones models, and learning from mentors in the operating theatre (OT).4 While these provide foundational knowledge and hands-on experience, each has its limitations. Technique guides and videos often require the learner to have a baseline understanding of the procedure. Cadaveric dissections are resource-intensive and are non-reusable. SawBones simulations neglect the crucial aspect of soft tissue handling and are non-reusable. Learning from surgeons in the OT can be challenging for residents who are seeing the procedure for the first time. There are also additional stressors such as time constraints and operating on a real patient. Reduced residents’ operative exposure also limits the time spent apprenticing with expert surgeons.1–3
While prior multi-centre trials (such as those by Lohre et al. evaluating immersive VR platforms) have demonstrated the efficacy of virtual reality in orthopaedic education, the interplay between standalone simulation and traditional master-apprentice teaching remains heavily debated.7–9 Although commercial platforms like Osso VR provide standardised procedural modules, the specific value-add of combining human mentorship with closed-loop VR task repetition has not been fully mapped out for absolute novices. Providing an interactive environment, VR simulation programs are able to simulate surgical procedures within a realistic OT setting. Learners can engage with virtual instruments, perform procedures repeatedly, and hone their skills in a risk-free environment. This immersive experience enhances technical competency, procedural confidence and sharpens decision-making abilities. Many randomised controlled trials have demonstrated the superiority of VR simulation training over traditional learning methods, such as technique guides and videos, in coaching the learner to perform a surgical procedure.9–14
However, there are limited studies to evaluate if VR simulation training alone is effective in imparting surgical procedure knowledge. The traditional master-apprentice relationship has long been the gold standard of training for surgeons.15,16 There is, however, a shift from the traditional dogma of “see one, do one, teach one”, with increased emphasis on simulation training programs to teach and enhance surgical competency.17–19 As VR technology gains traction in orthopaedic training, we question whether mentors remain relevant in imparting orthopaedic procedural knowledge in this era of technological advancement.9–14 With the increasing integration of VR into surgical education, evaluating whether it could supplant traditional mentoring, or better act as a complementary learning tool, is pertinent. Therefore, this study aims to compare the procedural competency of the learner when trained by either a mentor or VR simulation alone, or both, in performing femoral intramedullary nailing (IMN) on a SawBones model. We hypothesised that the combination of mentor-guided training and VR simulation will result in a significantly higher procedural competency in performing femoral IMN compared to either mentor-only or VR-only training.
METHODS
First- and second-year (pre-clinical) medical students from the Yong Loo Lin School of Medicine, Singapore, were recruited for the study. Students in their clinical years and orthopaedic residents were excluded due to potential exposure to previous femoral IMN procedures. An a priori power analysis conducted using G*Power (version 3.1) determined that a total sample size of 30 participants (10 per arm) was required to detect a large effect size (f = 0.50 or Cohen’s d = 0.80) in a one-way ANOVA, assuming an alpha of 0.05 and a statistical power of 0.80, based on variance parameters from prior surgical simulation literature.4,13,20,21 To account for anticipated attrition and exclusion criteria, 40 participants were initially recruited. This randomised controlled trial was conducted following approval by the National Healthcare Group Domain Specific Review Board. Written informed consent was voluntarily obtained from all participating medical students before any study-related procedures. Four students with previous exposure to either femoral IMN operation, orthopaedic surgical instruments, SawBones, or VR-based gaming systems were excluded. Three students declined to participate subsequently due to scheduling conflicts. The remaining 33 students completed a screening questionnaire before the study day to rule out any pre-existing conditions that would exclude them from the study (Appendix 1). Three students were excluded due to a history of motion sickness or car sickness, leaving a total of 30 students who participated in the study (Figure 1).
On the study day, all 30 participants first observed an orthopaedic consultant demonstrate the safe use of a surgical drill, including inserting and removing different-sized drill bits, chuck mechanism use, and advancing and reversing the drill. The consultant ensured that each student demonstrated uniform competency and safety before they could proceed. They also completed a survey to confirm their lack of prior experience with the insertion of a short femoral IMN (Synthes TFN-Advanced Proximal Femoral Nailing System) in an intact proximal femur SawBones model (Appendix 2).
Participants were randomised in a 1:1:1 ratio using computer-generated random numbers enclosed in sequentially numbered, opaque, sealed envelopes to ensure allocation concealment. The randomisation sequence was generated by an independent researcher not involved in participant recruitment or assessment (Figure 1). In the mentor-only group, a fellowship-trained orthopaedic consultant conducted a 20-minute live demonstration of inserting a short femoral IMN in a SawBones model. In the VR-only group, participants were first introduced to the commercial Osso VR platform (Palo Alto, CA, USA) utilising standardised hardware (Oculus Quest 2 headset and Touch Motion controllers). They were taught to use the Oculus Touch Motion controller until they were comfortable with it. They completed a step-by-step VR training module in tutorial mode for inserting a short femoral IMN, with instructions and prompts displayed for each step (Figure 2). After completing the tutorial, they progressed to the test module, where instructions and prompts were only provided when requested. They had to complete the test module once within 20 minutes. For the combined group, participants received mentorship training followed by VR training. The surgical steps taught and instrument layout in both mentor and VR groups were the same. To maintain uniformity, participants were not allowed to ask questions about the procedure, and groups were kept separated throughout the study.
Following their assigned training, the participants were escorted to a separate room to perform the insertion of a short femoral IMN in an intact proximal femur SawBones model. They had 20 minutes to perform the procedure using instruments arranged identically to their training. Two fellowship-trained orthopaedic consultants, blinded to the participants’ assigned training groups, independently evaluated each procedure using the Global Assessment Score (GAS) and a procedure-specific checklist. Inter-rater reliability between the two evaluators was quantified using the intraclass correlation coefficient (ICC) before final data analysis (Tables 1 and 2).4 If a student encountered difficulties and could not proceed, they received a score of 0 for that step. They were told to stop if the procedure could not be completed within 20 minutes.
All collected data were blinded and analysed using IBM SPSS Statistics, Version 25 (IBM Corp, Armonk, NY, USA). Statistical analysis was performed using one-way ANOVA with significance defined as p < 0.05 based on Geisser-Greenhouse correction for multiple comparisons. Kruskal-Wallis, Mann-Whitney U, and chi-square tests were used as appropriate for non-parametric, ordinal or categorical variables.
RESULTS
For the GAS, the combined group received significantly higher scores across all categories, outperforming both the mentor-only and VR-only groups (combined group: mean 13.2 ± 1.3, mentor-only group: 9.7 ± 1.2, VR-only group: 5.4 ± 1.1; p < 0.05 following Bonferroni correction for multiple comparisons). (Table 3). Participants in the mentor-only group also received significantly higher scores across all categories when compared to the VR-only group (p<0.05).
The procedure-specific checklist assessment showed that participants in the combined group completed 17.5% more steps correctly than the mentor-only group, and 40% more steps than the VR-only group, of which both were statistically significant (combined group: 42.5% correct steps, mentor-only group: 25% correct steps, VR-only group: 2.5% correct steps, p<0.05). Participants in the mentor-only group completed 22.5% more steps correctly compared to the VR-only group, which was also statistically significant (p<0.05) (Table 4).
Students in the combined group, compared to the mentor-only and VR-only groups, significantly performed better for 4 out of 8 steps (guidewire setup, nail assembly, nail insertion and helical blade insertion) (p<0.05). The percentage completed for the remaining 4 steps across all groups (open canal, alignment check, proximal locking and distal locking) was not statistically significant. (Table 4). When comparing between the mentor-only and VR-only groups, apart from nail insertion, the difference in performance for the other steps was not statistically significant (p>0.05).
DISCUSSION
The master-apprentice relationship has long been the cornerstone of surgical training. However, there is growing emphasis on simulation programs to teach and enhance surgical competency.17–19 Thus, we question if mentors remain relevant in imparting orthopaedic procedural knowledge in this “brave new world” of technology. Our study aimed to evaluate this by comparing the procedural competency of inexperienced medical students performing the insertion of a short femoral IMN after being trained by either a mentor or VR simulation alone, or a combination of both.
The combined group demonstrated significantly better overall technical proficiency based on the GAS (combined group: 13.2, mentor-only: 9.7, VR-only: 5.4, p<0.05). They also correctly completed a significantly higher percentage of steps than those in the other training groups (combined group: 42.5%, mentor-only group: 25%, VR-only group: 2.5%, p<0.05). The combined group likely benefited from task repetition, an established training methodology.22,23 Markose et al. emphasised the importance of task repetition in reinforcing neural pathways in surgical training, showing that accumulated experience improves outcomes.22 Even with task repetition over a short period, the combined group significantly outperformed the others. Gustafsson et al previously found that inexperienced orthopaedic residents required an average of 169 minutes of VR training before confidently performing supervised hip fracture surgery using cannulated and sliding hip screws.24 Further evaluation is required to determine if task repetition using VR alone can be successful in teaching orthopaedic surgical procedures, particularly those with more complex steps, and the training duration required to achieve proficiency.
Our analysis revealed that the mentor-only group performed significantly better for the GAS (mentor-only group 9.7, VR-only group: 5.4, p<0.05), and correctly completed 22.5% more steps than the VR-only group. The adage, “see one, do one, teach one”, emphasises the effectiveness of observing a demonstration before performing a procedure, rather than just being instructed to perform it.25–27 In the VR simulation program, there is no demonstration of the procedure in either the tutorial or test module before the learner performs it, leaving them unprepared and unable to fully benefit from the program. In contrast, participants in the mentor-only group benefited from observing a live demonstration, which provided a vivid and long-lasting impression that improved their competency.25 This underscores the relevance of mentors in demonstrating a procedure, something that VR training alone could not achieve in this study. Future VR simulation programs should incorporate videos of surgical procedures being performed before instructing learners to execute the steps. Continuously evaluating future VR simulation programs for orthopaedic procedures is crucial to determine their use in improving orthopaedic trainees’ surgical competency.
Although VR technology has revolutionised surgical training by introducing dynamic methods for skill development and evaluation, the traditional apprenticeship model remains crucial. Sommer et al. questioned whether simulator performance translates to actual surgical skills in the OT, as VR simulation cannot fully replicate the experience of performing surgery on a real patient, in a real-life OT environment. This is due to procedural complexities associated with patient-specific characteristics, such as anatomy, general condition, surgical and medical history, and co-morbidities.28 While live mentorship remains the gold standard for conveying nuanced intraoperative decision-making, exclusive reliance on one-on-one expert instruction is increasingly unsustainable due to duty-hour restrictions and consultant productivity pressures. Conversely, while VR cannot yet fully replicate unpredictable anatomical variations or real-time soft-tissue handling, our findings demonstrate that it serves as an exceptional scaffolding tool. By coupling a mentor’s initial qualitative demonstration with the high-repetition capacity of VR, programs can optimise faculty time while accelerating technical proficiency among learners. We propose a training model where mentors first demonstrate a procedure to inexperienced trainees. Trainees should then utilise VR simulation to achieve task repetition until they reach a learning plateau.22,23,29 However, orthopaedic residency programs must assess the appropriateness and timing of integrating VR simulation into their training curricula, considering both educational outcomes and resource allocation.
This study has several notable limitations. First, the use of pre-clinical medical students without prior surgical experience limits the direct generalisability of these findings to orthopaedic surgical residents or practising surgeons. The small sample size (n=30), while powered for primary effect size detection, restricts subgroup analyses. The linear regression model evaluating sub-steps should be interpreted as exploratory rather than definitive due to the restricted sample size. The evaluation was restricted to a dry-lab SawBones model over a single session; future studies must assess skill retention over time and transferability to human cadaveric or clinical settings. The assessment of participants’ technical skills using the GAS is subjective, and variability among graders exists. We mitigated this by using a validated scoring tool.4 At the same time, this also reflects a real-world scenario where different faculties may not agree on a resident’s competency level. Participants in the mentor-only and VR-only groups were exposed to the procedure only once due to resource and time constraints. Ideally, the training time across all three groups should be the same. Nonetheless, we were still able to evaluate the outcome between the mentor-only and VR-only groups, and the effect of task repetition in the combined group. Future research should determine the optimal VR training duration to achieve surgical competency equivalent to mentor-led training.
CONCLUSION
In conclusion, combining live mentorship with virtual reality simulation significantly improves procedural acquisition for novice learners on a SawBones model compared to either modality alone. While VR serves as a valuable adjunct for task repetition, these findings highlight the continued necessity of foundational expert instruction, though further research is required to evaluate translation to clinical orthopaedic training.
CONFLICT OF INTEREST
On behalf of all authors, the corresponding author states that there is no conflict of interest.
ACKNOWLEDGMENTS
None.
FUNDING INFORMATION
No funds were received for this study.
