search

jcfr / ci-sandbox

My sandbox for experimenting with CI services
Apache License 2.0
1 stars 4 forks source link

Project: Integration of Haptic Device in 3D Slicer for Lumbar Puncture #40

Open jcfr opened 1 year ago

jcfr commented 1 year ago

Category

IGT and Training

Key Investigators

Project Description

The main objective of the project is to integrate the haptic device Touch 3D Systems into 3D Slicer through an OpenIGTLink connection with the Unity platform. Slicer To Touch is the 3D Slicer module that contains the scene with the 3D models of the spine and the needle. This module has an interface where the user can configure the number, position and value of the resistances to be exerted by the haptic device. These values will be included in a .json file that will later be transferred to Unity, which will process this data and configure the forces of the haptic device within the Unity environment. Finally, through the OpenIGTLink connection bridge, a real-time connection will be created, where the transformations and the resistances of the haptic device will be shared with the 3D Slicer scene. This idea comes from a project for a lumbar puncture training system that makes use of this device, but the body tissues, their location and thickness are generic. This way you can make a segmentation of a real patient's back with its own characteristics and practice the lumbar puncture before doing it with the patient. Due to the way it works, it could be used in other procedures.

Objective

  1. Create a module that with the help of Unity and OpenIGTLink allows us to interact with a back model of a real patient obtained by segmentation of medical images. In this way we can train the lumbar puncture on the model of a real patient feeling the resistance in the body tissues.
  2. Automate the process of creating resistances on segmentation-generated models so that clinicians can easily perform lumbar puncture and other procedures with a sense of realism.

Approach and Plan

  1. Creation of the 3D Slicer module with fields to enter the number of resistances, positions and values.
  2. Generate a .json file with all the information entered in the module.
  3. Create a Unity project with a script that reads the generated .json file and creates a scene with the resistances in that position and with those values.
  4. Connect Unity to 3D Slicer through OpenIGTLink to send the transforms and see the needle movements in 3DSlicer.
  5. Generate an executable application from the Unity project with a simple look and feel that does the procedure automatically so that the clinical user finds it easy to use and does not have to deal with the unity interface.
  6. Do a documentation search of other procedures to check that the project works correctly in them. We are looking for other clinical procedures for which this project may be useful and for which there is information a

Progress and Next Steps

  1. Creation of the 3D Slicer module with fields to enter the number of resistances, positions and values. (DONE)
  2. Generate a .json file with all the information entered in the module. (DONE)
  3. Create a Unity project that reads the generated .json file and creates a scene with the resistances in that position and with those values. (DONE)
  4. Connect Unity to 3D Slicer through OpenIGTLink. We are working on this step we are working on this step based on a NAMIC project link by Alicia Pose.

Illustrations

3D Slicer Module in which you enter the resistances (left) and the .json file with the information of these resistances (right)(Picture1.png)

Picture1

Unity interface after reading the information from the .json file with the resistances created in the positions and the needle as a visual mesh of the haptic device (left) and script that makes it work (right) (Picture2.png)

Picture2

Background and References

References

github-actions[bot] commented 1 year ago

Project Page Pull Request Creation

:white_check_mark: COMPLETED: See https://github.com/jcfr/ci-sandbox/pull/43