The growing interest of Augmented Reality (AR) together with the renaissance of Virtual Reality (VR) has opened new approaches and techniques on how professionals interact with medical imagery, plan, train and perform surgeries and also help people with special needs in Rehabilitation tasks. Indeed, many medical specialties already rely on 2D and 3D image data for diagnosis, surgical planning, surgical navigation, medical education or patient-clinician communication. However, the vast majority of current medical interfaces and interaction techniques continue unchanged, while the most innovative solutions have not unleashed the full potential of VR and AR. This is probably because extending conventional workstations to accommodate VR and AR interaction paradigms is not free of challenges. Notably, VR and AR-based workstations, besides having to render complex anatomical data in interactive frame rates, must promote proper anatomical insight, boost visual memory through seamless visual collaboration between professionals, free interaction from being seated at a desk (e.g., using mouse and keyboard) to adopt non-stationary postures and freely walk within a workspace, and must also support a fluid exchange of image data and 3D models as this fosters interesting discussions to solve clinical cases. Moreover, VR and AR-based techniques must also be designed according to good human-computer interaction principles since it is well known that medical professionals can be resistant to changes in their workflow. In this course, we will survey recent approaches to healthcare, including diagnosis, surgical training, planning, and followup as well as AR/MR/VR tools for patient rehabilitation. We discuss challenges, techniques, and principles in applying Extended Reality in these contexts and outline opportunities for future research.

Bio

Joaquim Jorge is a Professor at Instituto Superior Técnico (IST/UL), the School of Engineering of the University of Lisboa, Portugal, where he teaches User Interfaces and Computer Graphics. He received PhD and MSc degrees in Computer Science from Rensselaer Polytechnic Institute, Troy, NY, in 1995. He is Editor in Chief of Computers & Graphics Journal and a was a member of the ERCIM Editorial Board from 2011 to 2015. He is currently Chair of the ACM/SIGGRAPH Specialized Conferences Committee, and a member of the IEEE Visualization and Graphics Technical Committee and Eurographics. He was a member of the Executive Committee of the ACM Europe Council from 2015-2019. He has organized or helped organize 45+ International Scientific Events. He served as International Program Committee Chair of Eurographics 2016, the 2013 International Conference on CAD/Computer Graphics (HK/PRC), ACM IUI 2012, among others. He has served on 200 International Program Committees. Joaquim Jorge’s interests are in Virtual and Augmented Reality, Visual Languages and Pattern Recognition techniques applied to Human-Computer Interaction. A Senior Member of IEEE (2001) and a Distinguished Member of the ACM (2017), he was elected Fellow of the Eurographics Association in 2010 and received the IFIP Silver Core Award in 2014 for services to TC13. He is an ACM Distinguished Speaker since July 2015.