Only just released to the public, Apple’s new Vision Pro headset can already be used to display medical records, filling a doctor’s field of view with anything from graphs of a patient’s blood pressure over time to the results of their latest chest X-ray.
The expensive new technology, with its ultra high-resolution screens for each eye and multiple cameras for hand and eye tracking, brings a new level of precision to a realm that, up to this point, has mainly been the domain of video games.
It remains to be seen whether Apple’s latest innovations will have what it takes to usher in a new era of virtual reality in health care, but the new Spatial Computing Center of Excellence now getting underway at Sharp HealthCare in San Diego aims to find out.
Sharp just took delivery of 30 Vision Pro sets, giving them to health care workers doing all sorts of jobs.
Apple is pitching this new product as a spatial computing device, one capable of projecting one’s digital work and entertainment across a broad virtual landscape, either transporting a person to a digital domain or overlaying information on a person’s immediate surroundings.
Sharp is working with Epic, the industry leader in electronic health records systems, to explore how Vision Pro might be used in health care settings.
Dr. Tommy Korn demonstrates a new program Sharp Healthcare is launching to use virtual and augmented reality to help train health care workers.
(Alejandro Tamayo / The San Diego Union-Tribune)
Sure, noted Dan Exley, Sharp’s vice president of clinical systems, VR headsets allow people to view information differently than they have in the past, but there is a big difference between different and better.
Sharp’s focus, he said, will be doing rigorous comparisons of old and new ways of getting health care work done, trying to identify which tasks prove better on a headset.
“We have invested in enough devices so that, right away, we can have physicians and nurses and informaticists and software developers and others start using it,” Exley said. “We want them to work with us to figure out which tasks and workflows it’s best for.”
There are some areas that just seem obvious.
Vision Pro uses internal cameras pointed at its user’s eyes to track their gaze, allowing them to select icons and other virtual objects by simply looking at them. This precise eye tracking suggests that the device might be very good at conducting patient eye exams in family medicine exam rooms.
But this technology is flexible enough that many are already suggesting less obvious ways it might be used. Tasks where a single person must monitor myriad situations seem like a good place to start. Nursing managers, for example, might be able to better visualize the vital stats of all the patients assigned to nurses they oversee. Or workers assigned to monitor video feeds of patients at risk of getting out of bed and falling in their rooms might be better off keeping watch with a headset, sweeping their gaze to feeds that need further investigation.
“You know, anesthesiologists monitor a ton of information in real time, vital signs, EKG, spirometry, waveforms,” Exley said.
One idea is to put those readouts into the headset and have them appear around an anesthetized patient’s head if the headset was set to use its outward-facing cameras to pass through a view of the real world, allowing information to be overlaid on top.
“Now you’re able to just keep your eyes on the patient, you’re not looking back constantly to see what the monitors are saying behind you,” Exley said.
Virtual reality has already made significant inroads into health care. It is a common technology used to deliver exposure therapy to treat post-traumatic stress disorder, allowing patients to experience traumatic “triggers,” gradually causing stress reactions to become less intense.
Surgical teams are also starting to use augmented reality, overlapping digital images and real-world scenes to enhance medical procedures. UC San Diego Health began such operations in 2021, using the technology in complex spinal operations to help project where mechanical fasteners would need to be placed, allowing surgeries to be performed more quickly.
Studies are also beginning to show that virtual models, which can be shown in three dimensions, are great for teaching medical professionals complex anatomy and for helping patients better understand medical procedures they will experience.
On Monday morning at Sharp’s new innovation center in Kearny Mesa, Dr. Tommy Korn delighted in bringing up a three-dimensional model of a human heart, rotating the digital muscle to show its different chambers and even how blood flows through different valves.
Patients might wonder whether they are soon to see their doctor come into their exam room wearing a digital headset, but Korn said wearing a headset during interactions with patients generally doesn’t make sense because doing so would reduce the amount of human interaction. But, using VR to get a deeper look at a patient’s records before entering the exam room could be a boon.
“Now I’m armed, before I go into the exam room, with all of the knowledge organized visually and conveniently,” Korn said. “Now I can share that information with the patient and not be focused on the computer.”
