🧲 Op MRI (Duke and Prostate-X)

[SagiPolaczek]

The purpose of this PR is to finish Michal's (Tal's) work on the MRI Ops and the Duke & ProstateX examples.

PR IS NOT READY

Plan:

• [x] fix code style (format and type hints - black, flake, mypy)
• [x] make CI pass
• [x] duke dataset
• [x] duke runner, using PyTorch Lightning. state: runner run OK, but fail in sub process test.
• [x] prostate dataset
• [x] prostate runner, using PyTorch Lightning (Michal's works)
• [ ] trim the code (after all is running OK)
• [ ] detect issues and debugging them

Issues:

• [ ] tests take too much time. currently skip the runners in the CI.

[SagiPolaczek]

I added a draft for visualizing 3D images. Firstly I tried this one, but it didn't work that good thus I quickly switched to viewing slices in the following way:

Another interesting option is to save a GIF of the slices. Any thoughts? :) @mosheraboh

[mosheraboh]

Thanks, Sagi! I told also to @smartdanny to think about options for 3D visualizer. @smartdanny, see if there is something interesting we can add in addition to the basic 3D visualizer Sagi added (see his last comment)

[shatz01]

Hey thanks for the mention @mosheraboh :). Ive had some good (albeit basic) experience with the ipyvolume package: https://www.youtube.com/watch?v=hOKa8klJPyo

Have you guys considered it for this issue? @SagiPolaczek

[SagiPolaczek]

Hey thanks for the mention @mosheraboh :). Ive had some good (albeit basic) experience with the ipyvolume package: https://www.youtube.com/watch?v=hOKa8klJPyo

Have you guys considered it for this issue? @SagiPolaczek

Actually I'm not familiar with it, seems cool :)

Is it applicable outside a Jupyter session? If so in my opinion such a 3d plot will be interesting in MRI context.

[shatz01]

Yea looks like it may be difficult/impossible to get ipyvolume working outside of jupyter.

In any case, can you tell me where exactly you developed and how you are debugging this 3d visualization? Are you running it in one of the examples? Im a little lost haha 😅 @SagiPolaczek

[SagiPolaczek]

@smartdanny Sure:

So yeah, I'm running it on the ProstateX example (on this branch).

The visualizer is just part of the data pipeline: (in fuseimg/datasets/prostate_x.py ProstateX.dynamic_pipeline())

            (
                OpVis3DImage(num_samples=1, show=False),
                dict(key="data.debug.3d_volume", n_rows=2, n_cols=3, channel_axis=0),
            ),

To run the example itself you should run (for example):

python examples/fuse_examples/imaging/classification/prostate_x/runner_prostate_x.py --heldout_fold=4

but first you should set an env variable for the prostateX dataset. contact me in slack for that one (if you're interested) 😄

[shatz01]

Hey I managed to get this working in the 3dvisop: https://plotly.com/python/visualizing-mri-volume-slices/

And was able to export to html

Then from vscode you can download the html and view it https://user-images.githubusercontent.com/22205363/197400979-298f98cc-a0e7-44df-928b-93d4221224a0.MOV

What do you think?

[SagiPolaczek]

@smartdanny

Looks really nice in my opinion :) Could you please push it to a new PR so I can play with it? Or paste here the relevant code if its shorter

[SagiPolaczek]

Cleaning PRs. I'm not deleting branch - for future ref.