From the disclosure of x- rays in 1895 to the use of ultrasound, MRI, and CT scanners in the 1960s and 70s, health care has long relied upon imaging to provide the best quality diagnostics and care to patients. Presently, as new forms of imaging such as virtual reality and augmented reality hit the scene, analysts including Jayender Jagadeesan, Ph.D., of the Surgical Planning Lab in the Radiology Department at BWH, hope to bring these imaging techniques into the operating room.
Immersive technologies mix this real world with digitally simulated conditions to transport the user into a virtual, augmented or mixed reality. Despite the fact that all slightly unique, each type of “reality” opens the door for brand new applications in medicinal imaging.
Jagadeesan said, “There is a lot of data and new technology available for doctors. The question is, how would we use these images and data in real-time during surgery without going into data overload?”
Enveloped in a Digital World
As the most immersive type of imaging, virtual reality (VR) takes the user into a totally digital world with digital objects. Jagadeesan and his associates at the National Center for Image-Guided Therapy (NCIGT) at BWH see virtual reality as a potential tool for use in diagnostics and education.
The NCIGT is an NIH-financed center led by Clare Tempany, MD whose objective is to improve medical procedures utilizing patient-specific imaging during surgery or treatment. In radiology, it is difficult to look over several scans to spot minute changes in small masses in the lungs after some time to determine whether they are cancerous. In any case, with a wrap around VR headset such as the Oculus Rift, a radiologist could project various images all the while over their vision, making it simpler to distinguish changes.
VR could also enable doctors to work together on diagnostics from remote locations. They would be able to view similar scans in the meantime and utilize a digital rendering of their hands to point out different areas on the scans. This shared virtual reality could also be utilized as a tool to educate new physicians.
With mixed reality (MR), another type of immersive technology, the user sees digital objects superimposed onto a digital rendering of this present reality. At the NCIGT, scientists are testing the utilization of MR in laparoscopic surgery. Analysts combine data from webcams, the laparoscope, the navigation system and 3D preoperative imaging and make the information all visible through the Oculus Rift.
Up until now, they have utilized MR only in pre-clinical studies to show the possibility of utilizing mixed reality in laparoscopic surgeries, but they plan to start utilizing it for lymph node removal surgeries in collaboration with Ali Tavakkoli, MD, later this year.
Clinical teams are additionally exploring applications for augmented reality (AR), the process of taking digital images and overlaying them over this present reality. With the popularity of games like Pokémon Go, most smartphone users are now familiar with this kind of imaging.
Augmented reality could be used for surgical planning preceding an operation. Utilizing the HoloLens, a wearable, head-mounted AR device, specialists can generate 3D models of the patient’s anatomy to create a plan for the best approach to surgery.
Jagadeesan and the NCIGT are hoping to utilize the HoloLens for spinal tumor ablations (with Tom Lee, MD), skull-base surgeries (with Eduardo Corrales, MD), neurosurgeries (with Alex Golby, MD) and sarcoma resections (with Raut) and may begin selecting patients in the near future. The future of immersive technology in medicine is continually extending.
Jagadeesan said, “This is only the start of what could be possible with these new tools. Improving our reality with immersive technologies is an exciting and promising way to further advance what’s possible in medicine.”