First experiments with the metaverse? See what’s going on in medicine

Surgeon Shafi Ahmed poses for a photographer wearing a Microsoft HoloLens headset in his operating room at Royal London Hospital on Thursday, January 11, 2018.

Bloomberg | Bloomberg | Getty Images

The metaverse, the next big thing in the digital world, is being touted as an internet domain where animated avatars of our physical bodies can one day virtually engage in all kinds of interactions, from shopping to gaming and travel. The wonky say it could be a decade or more before the necessary technology catches up with the hype.

However, right now, the healthcare industry is using some of the core components that will eventually include the metaverse – virtual reality (VR), augmented reality (AR), mixed reality (MR), and artificial intelligence (AI) – as well as software and hardware. for their applications to work. For example, medical device manufacturers use MRI to assemble surgical instruments and design operating rooms, the World Health Organization (WHO) uses augmented reality and smartphones to educate Covid-19 responders, and psychiatrists use virtual reality to treat post-traumatic stress disorder (PTSD). among combat soldiers and medical schools are using VR for surgical training.

Facebook, Oculus and Covid

With Facebook – now Meta Platforms – Acquired Oculus and its VR headset technology in 2014 for $ 2 billion, numerous healthcare apps have been developed. Among the most recent have been collaborations with Facebook Reality Labs, Nexus Studios and the WHO Academy. The organization’s research and development incubator has developed a mobile educational app for healthcare professionals fighting Covid-19 around the world. One of the training courses includes AR for simulating the correct techniques and sequence of putting on and taking off personal protective equipment on a smartphone. The app, with content available in seven languages, is based on the needs expressed by 22,000 global health professionals surveyed by WHO last year.

Oculus technology is being used at UConn Health, the University of Connecticut Medical Center in Farmington, Connecticut, to train orthopedic surgeons. Educators have teamed up with PrecisionOS, a Canadian medical software company that offers orthopedics VR training and education modules. By wearing Oculus Quest headsets, residents can visualize a range of surgical procedures in 3D, such as poking a pin into a broken bone. Since the procedure is performed virtually, the system allows students to make mistakes and get feedback from teachers, which they can use on their next attempt.

Meanwhile, with the metaverse still under construction, “we see a great opportunity to continue the work that Meta is already doing to support the healthcare effort,” a Meta spokesman said. “As Meta’s experience, applications and services evolve, the healthcare strategy can be expected to play a role, but it is too early to tell how this might overlap with third-party technologies and vendors.”

When Microsoft unveiled its HoloLens AR smart glasses for commercial development in 2016, early adopters were Stryker, a medical technology company based in Kalamazoo, Michigan. In 2017, the company began using the AR device to improve operating room design processes for hospitals and surgery centers. Because operating rooms are used by a variety of surgical services – from general surgery to orthopedic, cardiac, and others – lighting, equipment, and surgical instruments differ depending on the procedure.

Recognizing the opportunity the HoloLens 2 has presented in moving operating room design from 2D to 3D, Stryker engineers can design common operating rooms using holograms. The MR experience visualizes all people, equipment and installations without requiring the presence of physical objects or people.

Zimmer Biomet, a medical device company based in Warsaw, Indiana, recently unveiled its OptiVu Mixed Reality Solutions platform, which uses HoloLens devices and three applications: one uses MRI to make surgical instruments, and the other collects and stores data to track patient progress. before and after it. after surgery, and a third that allows clinicians to share the MRI experience with patients before the procedure.

“We are currently piloting HoloLens with remote assistance in the US, Europe, the Middle East and Africa and Australia,” said a Zimmer Biomet spokesman. The technology has been used for remote patient monitoring and education programs, and the company is developing software applications for HoloLens as part of its pre- and post-treatment data solutions, the spokesman said.

Microsoft’s holographic vision of the future

In March, Microsoft unveiled Mesh, an MR platform powered by the Azure cloud service that allows people in multiple physical locations to join a 3D holographic experience across a variety of devices, including HoloLens 2, a range of VR headsets, smartphones, tablets and PCs. … On its blog, the company featured avatars of human anatomy medical students gathered around a holographic model and tearing apart their back muscles to see what was underneath.

Microsoft sees many opportunities for its MR technology, and in March provided a $ 20 billion contract with the US military to use it with soldiers.

In the practical application of medical technologies of augmented reality, neurosurgeons at Johns Hopkins performed the first ever surgical operations on living patients in June. During the initial procedure, doctors inserted six screws into the patient’s spine during fusion. Two days later, a separate team of surgeons removed the cancerous tumor from the patient’s spine. Both teams donned headsets from the Israeli firm Augmedics, equipped with a transparent eye display that projects images of the patient’s internal anatomy, such as bones and other tissues, based on computed tomography. “It’s like having a GPS in front of your eyes,” said Timothy Whitham, MD, director of the Johns Hopkins Spine Neurosurgery Laboratory.

At the University of Miami Miller School of Medicine, instructors at the Gordon Center for Modeling and Innovation in Medical Education use AR, VR and MR to train emergency care professionals in the treatment of trauma patients, including those with a stroke, heart attack, or gunshot wound. Students practice life-saving cardiac procedures on the Harvey, a life-like dummy that realistically simulates virtually any heart condition. In VR headsets, students can “see” anatomy, which is displayed graphically on Harvey.

“In the digital environment, we are not connected by physical objects,” said Barry Issenberg, MD, professor of medicine and director of the Gordon Center. Prior to developing the virtual technology curriculum, he said, students had to be physically present on stage and trained with real trauma patients. “We can now ensure that all students experience the same virtual experience, regardless of their geographic location.”

Since its founding in 1999, the University of Southern California Institute for Creative Technology (ICT) has developed virtual reality, artificial intelligence and other technologies to treat a variety of diseases and mental health. “When I first got involved, the technology was stone age,” said Albert “Skip” Rizzo, psychologist and director of medical virtual reality at ICT, recalling how he fiddled with the Apple IIe and the Game Boy handheld. Today he uses VR and AR headsets from Oculus, HP, and Magic Leap.

Rizzo helped create a virtual reality exposure therapy called Bravemind, aimed at alleviating post-traumatic stress disorder, especially among veterans of the wars in Iraq and Afghanistan. During exposure therapy, the patient, under the guidance of a qualified therapist, confronts his memories of trauma, simulating his experiences. With the headset on, the patient can immerse themselves in several different virtual scenarios, including a Middle East themed city and a desert road.

“Patients use keyboards to simulate people, rebels, explosions, even smells and vibrations,” Rizzo said. And instead of relying solely on the imagination of a specific scenario, the patient can experience it in a safe virtual world as an alternative to traditional conversational therapy. Bravemind’s evidence-based therapy is now available in over a dozen FDA hospitals, where it has been shown to significantly reduce PTSD symptoms. Additional randomized controlled trials are ongoing.

As Big Tech continues to build the metaverse with software and hardware companies, academia, and other R&D partners, the healthcare industry remains a real testing ground. “While the metaverse is still in its infancy, it has tremendous potential to transform and improve healthcare,” wrote Paulo Pineiro, head of software at Cambridge, UK-based Sagentia Innovation, on the consultant’s website. “It will be interesting to watch the development of the situation.”

Stay connected with Healthy Returns

By taking a front row seat at CNBC Events, you can directly hear the views of visionary executives, innovators, leaders and influencers taking the stage. “Keynote podcast… ” Now listenhowever you get your podcasts.

To learn more from our reporters and presenters, subscribe to our Healthy Returns newsletter to receive the latest news directly to your inbox on a weekly basis.

Source link

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button