21st Century Revolutionizing Cancer Detection with AR Microscope Driven by AI-
Google and the US Department of Defense’s Remarkable Collaboration
Disclaimer: The credit for this article goes to Google Research, Google Health, and CNBC
Written by – Rajeshwar Raj
Google and the Department of Defense have joined forces in a groundbreaking collaboration to develop an innovative Augmented Reality Microscope (AR microscope), a powerful tool that leverages artificial intelligence (AI) and augmented reality (AR) to enhance the detection of cancerous tumors. This cutting-edge technology, still in its early stages, holds immense promise in medical diagnostics.
The AR microscope closely resembles a traditional microscope, with the potential to revolutionize how pathologists diagnose cancer. Traditionally, pathologists rely on tissue samples to determine whether a patient has cancer, a responsibility with profound implications for patients’ lives. Dr. Niels Olson, Chief Medical Officer of the Defense Innovation Unit, sheds light on the challenges pathologists face in making these life-altering diagnoses. “It’s not just about identifying cancer types,” Dr. Olson explains. “Part of the job is ruling out cancer, and this pressure to make accurate diagnoses is immense. 
Concepts of the “AR microscope” originated with Dr. Olson during his 2016 assignment at the Naval Medical Center in San Diego. His idea gained traction after a fortuitous conversation with a contact at Google, leading to a demonstration of an early “AI-equipped microscope” at Google’s Mountain View, California office. The prototype showcased an impressive ability to accurately detect cancer, generating excitement among Dr. Olson and colleagues.
Subsequently, the Defense Innovation Unit assumed control of the AI-equipped microscope project, collaborating with Google and optical company Jenoptik to develop the AR microscope. In a 2019 article by Dr. Olson, he highlighted the immense potential of AI-powered microscopes, especially within the context of the U.S. federal healthcare system’s extensive medical data, often touted as the “world’s most comprehensive medical data set.” The integration of AI and big data promises significant advances in healthcare.
Although the AR microscope is not fully actively used for patient diagnosis, 13 of these cutting-edge devices have already been deployed at related facilities. These AR microscopes resemble traditional optical microscopes used in school science labs, complete with an eyepiece and a slide tray. However, they also incorporate a box-shaped computer tower housing the AI model. When a glass slide is placed under the microscope, the AI overlays a green frame onto the image from the eyepiece to indicate the cancer’s location and generates a pix-elated heat map outlining the tumor’s boundaries.
In recent years, there have been efforts to digitize the work of pathologists, but the sheer volume of data required for digitizing a single slide has proven to be a barrier. It has resulted in increased infrastructure costs and challenges for smaller healthcare institutions. The AR microscope offers a cost-effective solution by seamlessly integrating with traditional optical microscopes, reducing the burden on medical institutions’ pricing of $90,000 to $100,000.
The AI’s ability to generate a black-and-white heat map depicting cancer boundaries on a monitor provides valuable support for pathologists. Dr. Niels Olson emphasizes that the AR microscope is designed to complement, not replace, pathologists, serving as a second line of defense. Its initial application is anticipated in smaller, remote labs and may prove invaluable for pathology residents in training.
Google Cloud, in collaboration with the Defense Innovation Unit, has developed four algorithms for the AR microscope, enabling it to identify breast cancer, cervical cancer, prostate cancer, and mitosis. It is important that the data used to train these AI models remains secure, with neither Google employees nor Google infrastructure having access to it.
The Defense Innovation Unit aims to make the AR microscope accessible to all government users through the General Services Administration website in the near future. With its potential to transform cancer diagnosis and enhance the accuracy of pathology, the AR microscope represents a remarkable leap forward in medical technology. While its journey from conception to widespread adoption is ongoing, its promise for the healthcare industry is undeniable.
We express profound gratitude to the following Titans for bestowing upon us their invaluable contributions.
(Augmented Reality) -AR Microscope Driven by AI for Cancer Detection