|
Combining vision and touch in augmented reality to support surgical teams in reconstructing complex fractures. This is the focus of a team of researchers from the University of Pisa, who have developed a first prototype tested on a displaced knee fracture. A simple, low-cost haptic device created at the University's laboratories complements the augmented reality headset by combining visual and tactile stimuli. It is made up of two rings that are worn on the fingers to provide vibrotactile stimuli. These stimuli offer surgeons valuable information that enables them to accurately manipulate virtual bone fragments through natural gestures, including pressing, grasping, and aligning. “For the surgeon", explain the researchers, “it is not always easy to understand how to arrange the fragments of a fracture. Our method simulates the fracture in augmented reality using vision and tactile inputs, assisting the surgeon in planning the procedure". The experts further explain, "The device enhances spatial awareness and immersion during surgical training or preoperative planning by providing haptic feedback via vibration. The absence of haptic feedback is a substantial constraint in virtual and augmented reality surgical training systems, as medical personnel frequently rely on hand feel rather than sight in normal clinical practice. This is especially critical when working with delicate and hidden anatomical structures. The findings show that haptic feedback greatly increases accuracy and efficiency in fracture fragment reduction activities performed in virtual and augmented reality environments". The integration of vision and touch will therefore be fundamental in future augmented reality–based surgical training and assistance systems.
|