In line with medical device regulations, Human Factors should be applied to the development of safety-critical medical devices to validate their safety and efficacy in the hands of intended users.
In the case of complex medical systems, such as a robotic surgical system, the planning and execution of such studies require much more thinking, organisation, and resources. Bearing this in mind, we have grouped the effort involved in validating complex medical systems into four categories, providing guidance based on experience.
Learn from the experts
Sticking with the example of a robotic surgical system, there may be four distinct user groups to be considered; surgeon, surgical first assistant, scrub (sterile) nurse, and circulating (unsterile) nurse. In the lead up to the study, it is sensible to invest time in understanding the different user groups’ workflows and responsibilities so that task performance can be rated against the correct user, and use problems are not duplicated in the dataset.
Prior to the study, it is important to research any specific collective behaviors and unwritten rules. For example, a team may that decide bumping into an object with someone’s back might not be considered a break in sterility. Group huddles and other collective decision-making techniques could be used prior to surgery to achieve an appropriate level of consensus around “unwritten rules” prior to surgery. Furthermore, capturing a group huddle may provide supporting evidence for regulatory justifications where task success and acceptability is down to ‘clinical judgment’.
Human Factors professionals are not clinicians; therefore, it is advised to include experts in the field (known as preceptors) in the study. A preceptor can help determine if participants’ decision on how to perform a task during simulated use was clinically acceptable. Build in time to liaise with the preceptor before and after the study sessions and communicate the scope of their role.
Avoid scope creep
To make the most of the allocated resources, it may be tempting to execute multiple studies in one – for example, including a formative study in parallel on a specific aspect of the user interface. Although combining studies can be efficient, objectives can become entangled, and outcomes obscured can be obscured and lose clarity. If it is the case that multiple studies are happening in parallel, it is advantageous to split study protocols into individual documents containing well-specified objectives. This should help to define each activity to avoid scope creep as much as possible.Let us assume that the main activity is the usability validation of a robotic system with the groups we discussed previously; surgeon, surgical first assistant, scrub (sterile) nurse and circulating (unsterile) nurse. The protocol for the main study must include all critical tasks expected to be performed by these four user groups. Now, let us assume that a second activity is the usability validation of the reprocessing of robotic instruments. Reprocessing is a completely different process, performed by a distinct user group in a separate use environment. Therefore, writing independent study protocols will ensure the clarity of the two activities is not lost or entangled.
Post study, draft a separate report for each parallel activity to make it easier to confirm which objectives have been met and which ones, if any, have not been successful. Furthermore, independent protocols will provide the study team with clarity in the best interview techniques as approaches in Root Cause Analysis may differ depending on study objectives.
Pay attention to the details
Multiple observers will be common for complex studies where there are several users carrying out their own sequence of steps. Assigning an observer to each user will simplify the process. Consider name tags and a color-coding system to help identify participants when everyone is scrubbed in.
Given that complex studies may run over several days, it is important to have a detailed schedule. Consider how to optimize the schedule. For example, it might be beneficial to have the validation study on a day which is less intensive for the participants to reduce fatigue. Other scheduling considerations could involve building in training decay by planning a formative study, to assess a different component under development, after training is concluded.
Don’t forget the minutia: Who is synchronising clocks? Where will observers sit on the day? Who oversees video data management? What are the naming conventions? When answering these questions, ensure the workload is equally balanced throughout the team to prevent overloading a team member.
Be flexible
Consider how flexibility can be applied to safeguard the wellbeing of your study team. Validation and formative studies of any nature can be challenging and taking steps to support the physical and mental health of your team will lead to a more successful, data-rich study. Integrate breaks, have back up team members on standby and prepare for things to go wrong, which can often occur. It is not possible to perform a hysterectomy in a male body, or in a female cadaver who already had their uterus removed. You may need to deviate from the plan. Be flexible and at least simulate the critical tasks of the intended operation, while still respecting the body which has been donated to science.
Considering the complex nature of the operating room, you may also wish to be flexible in relation to the expected use process. There can be flexibility in these workflows, and it may not be an error for a user to assist their team members in task completion.
As the clinical applications for robotics assisted surgery expand, so too does innovation, iteration, and new entrants. Kinneir Dufort are here to support clients who need additional resources in a specific area or full life cycle development, to find out more, contact our Medical Business Development Manager, Michael Denver-Beautement, at michael.denver-beautement@kinneirdufort.com. You can also find all about our Human Factors capabilities here.
Please note, this article is based on our experience in validating complex system. The advice here should be used to compliment that validation process as stipulated by regulatory guidance. For more information on this process, please contact our Senior Human Factors Consultant, Molly Smyth, at molly.smyth@kinneirdufort.com
Explore more
