search

NA-MIC / ProjectWeek

Website for NA-MIC Project Weeks
https://projectweek.na-mic.org
84 stars 284 forks source link

Project: NVIDIA Holoscan and 3D Slicer #938

Closed brudfors closed 9 months ago

brudfors commented 9 months ago

Draft Status

Ready - team will start page creating immediately

Category

Infrastructure

Presenter Location

In-person

Key Investigators

Project Description

This project aims at integrating NVIDIA Holoscan into 3D Slicer. Holoscan is a hybrid computing platform for medical devices that combines hardware systems for low-latency sensor and network connectivity, optimized libraries for data processing and AI, and core microservices to run surgical video, ultrasound, medical imaging, and other applications anywhere, from embedded to edge to cloud.

Objective

Integrate Holoscan with 3D Slicer. As Holoscan is both a hardware platform (NVIDIA IGX Orin) and an SDK (Holoscan SDK) the integration could be of either of these components, or both.

Approach and Plan

There are various options, not yet sure which one is the best (or if there are ones I have not considered):

  1. Use OpenIGTLink to stream data from a Holoscan developer kit to 3D Slicer, over network. This would bypass the fact that 3D Slicer does not easily build on ARM (which is the arch of the IGX Orin).
  2. Create operators in 3D Slicer that can pass data to and from Holoscan SDK. This would give a 3D Slicer user access to the API of Holoscan SDK for efficient AI inference, etc.
  3. Connect an IGX Orin running Holoscan SDK to the Plus toolkit.

Progress and Next Steps

Illustrations

Background and References

pieper commented 9 months ago

This looks great @brudfors - look forward to chatting about the options. Add me to the project if you want.

RafaelPalomar commented 9 months ago

@brudfors, I'd be happy to join too. I'm glad you will join PW in person.

github-actions[bot] commented 9 months ago

Project Page Pull Request Creation

:white_check_mark: COMPLETED: See https://github.com/NA-MIC/ProjectWeek/pull/939

brudfors commented 9 months ago

Thanks @pieper and @RafaelPalomar! I will add you to the project then, looking forward to getting some of your input!