Augmented Reality has often been defined by its main difference to Virtual Reality in not requiring the user to immerse into a virtual world. However, AR systems have often required the users to use additional equipment and displays forcing them to immerse into a new environment for offering augmentations often only needed during short periods of time. The solution is to create user interfaces, which take advantage of AR only when required. One key to the success of such user interface is its ability to automatically recognize different phases of a workflow, which each require various levels of augmentation. It is also important for the AR system to be transparent to the user during the rest of the procedure. These issues have greater importance when dealing with computer-aided surgery applications. In most of these applications, a surgeon needs augmentation for only quite brief periods, such as choosing the ports for a laparoscopic intervention or localizing the major arteries before starting a liver resection. These augmentations, however, can play an important role in the overall procedure's success. During the past three years, the Chair for Computer Aided Medical Procedures (CAMP) has tried to develop such integrated AR solutions in the context of minimally invasive surgery. In this talk, I present an overview of these activities and our recent results. The focus is on recovering the workflow, modeling medical procedures, and intelligently integrating advanced data fusion, navigation and visualization into such procedures. We present different solutions for Trauma Surgery, Nuclear Medicine and Interventional Radiology applications. Phantom, ex-vivo and in-vivo experiments demonstrate the advantages of such new solutions.

Nassir Navab is a full professor and director of the institute for Computer Aided Medical Procedures and Augmented Reality (CAMP) at Technical University of Munich (http://campar.in.tum.de). He has also a secondary faculty appointment at the Medical School of TU Munich.

 

Before joining the Computer Science Department at TUM, he was a distinguished member at Siemens Corporate Research (SCR) in Princeton, New Jersey. He received the prestigious Siemens Inventor of the Year Award in 2001 for the body of his work in interventional imaging. He had received his PhD from INRIA and University of Paris XI in France and enjoyed two years of postdoctoral fellowship at MIT Media Laboratory before joining SCR in 1994.

 

He is a member of board of directors of MICCAI society and one of the first members of the steering committee of IEEE ISMAR.