Project: Augmented reality experiments for cardiology imaging

[pieper]

Draft Status

Ready - team will start page creating immediately

Category

VR/AR and Rendering

Presenter Location

In-person

Key Investigators

• Steve Pieper (Isomics, Inc., USA)

• Vitaliy Petrov (Lviv Regional Clinical Hospital, Ukraine)

Project Description

Explore tools for clinician review and patient education using smartphone browser-based augmented reality. There are two possible scenarios we could support:

• Using existing SlicerHeart and SlicerVirtualReality extensions to support smartphone tracking
• Exporting Slicer scene data to a web page for rendering locally in smartphone browser

Objective

1. Determine how much work would be required to make a system that could be tested in clinical scenarios
2. See what other people's experience has been in the feasibility and/or utility of such systems

A few of the features we'd like to explore:

• Using the a smartphone as a controller in SlicerVirtualReality
• Communicating events from smartphone to Slicer to control rendering
• Sending rendered images to phone vs. rendering in phone locally
• Exporting data to a stand-alone scene that could be viewed on a smartphone (e.g. by emailing a link or generating a QR code)

Approach and Plan

1. Review existing technologies for applicability for this use case
2. Talk with Project Week attendees about similar needs and experiences
3. If time, do some prototyping

Progress and Next Steps

1. Describe specific steps you have actually done.

Illustrations

No response

Background and References

• Phone based tracking communicating with Slicer from earlier Project Week: https://www.youtube.com/watch?v=kQKskHYlpQE
• Use of Slicer's WebServer to host tracker data: https://github.com/pieper/SlicerWeb/blob/master/WebServer/docroot/WebXR-controller/index.html
• vtk.js based volume rendering demo compatible with Android smartphone AR tracking: https://kitware.github.io/vtk-js/examples/WebXRVolume/WebXRVolume.html?xrSessionType=1&fileURL=https://data.kitware.com/api/v1/file/63fe3f237b0dfcc98f66a857/download&colorPreset=CT-AAA&rotateX=90&rotateY=180

[github-actions[bot]]

Project Page Pull Request Creation

:white_check_mark: COMPLETED: See https://github.com/NA-MIC/ProjectWeek/pull/956