Professor Raabe’s team from the University Clinic for neurosurgery at the University Hospital of Berne has worked with us to test new mixed reality applications in neurosurgery. One thing is already clear: The augmented reality approach provides better planning for surgical interventions.
A first specific field of application is the visualization of scan results and the corresponding planning of operations. Existing CT and MRI scans of patients and data from neuro-navigation can be divided into their component segments, colored, and visualized life-size in 3D. The spatial representation makes the situation more understandable and helps the surgeon in planning the operation.
To this end, we’ve developed an easy-to-use and intuitive app. The app runs directly on the HoloLens, no other devices are needed. The doctor loads the patient data directly into the HoloLens from the neuro-navigation system without further manual steps. The patient is selected using a menu in the HoloLens, then the scans are loaded directly into the glasses and visualized in 3D space. Various segments such as the head, blood vessel and individual areas of the brain are marked with colors and can be displayed or hidden. Segments can be resized and moved with gestures and voice commands.
At the beginning of the project, the situation was not clear. For example, we didn’t know whether the HoloLens is adequate for such complex models and whether the hospital’s 3D data could even be used for this purpose. We quickly proofed that the performance of the HoloLens delivers true added value together with the development tools and the existing data.
The solution is still in the prototype phase and not in medical use. In our opinion, such use is only a matter of time. We have already identified additional areas of application with further potential and included them in the planning.