gradio-app / gradio

Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work!
http://www.gradio.app
Apache License 2.0
33.41k stars 2.53k forks source link

Feature Request : Support for 3D medical images #4511

Closed blueyo0 closed 10 months ago

blueyo0 commented 1 year ago

Is your feature request related to a problem? Please describe.
I was wondering if it would be possible for Gradio to support 3D medical images such as NIFTI or DICOM files. It would be really great if Gradio could visualize 3D medical images! You can find some visualization examples in the OHIF Viewer.

Describe the solution you'd like
One kind of implementation of the ability is to convert the output 3D medical image (e.g., .nii.gz, .dcm) into multiple 2D image slices (e.g., *.jpg, etc). Additionally, it would be great if Gradio could support scrolling or wheeling to easily navigate between the different slices.

abidlabs commented 11 months ago

Hey! We've now made it possible for Gradio users to create their own custom components -- meaning that you can write some Python and JavaScript (Svelte), and publish it as a Gradio component. You can use it in your own Gradio apps, or share it so that anyone can use it in their Gradio apps. Here are some examples of custom Gradio components:

You can see the source code for those components by clicking the "Files" icon and then clicking "src". The complete source code for the backend and frontend is visible. In particular, its very fast if you want to build off an existing component. We've put together a Guide: https://www.gradio.app/guides/five-minute-guide, and we're happy to help. Hopefully this will help address this issue.

abidlabs commented 10 months ago

I'll go ahead and close this issue since we are not planning to include this in the core Gradio library. But happy to help if you are interested in making this a custom Gradio component (feel free to ask questions in this issue).

pangyuteng commented 10 months ago

@abidlabs thank you for providing the examples and tutorial link!

In the mean time, I've cooked up a barebones viewer using gr.AnnotatedImage for scrolling through axial slices ( abit laggy since we have client-server communication for every slider update): https://huggingface.co/spaces/sawtellelabs/papaya-image-viewer-gradio

There is also an alternative implementation by andreped below: https://huggingface.co/spaces/andreped/AeroPath

abidlabs commented 10 months ago

That's really cool! A custom component would be the ideal way to create a nice viewer, let us know if we can help