Add notebook example to launch data in Slicer and/or ohif

[pieper]

We have several analytics example notebooks in IDC that construct ohif urls for the database results. It would be interesting to try the same thing with slicer://.

An interesting scenario could be to make a TODO list for human annotators, something like this:

• for a given collection, segmentation dicomstore, and annotatorID
• query for all and display links for all studies in the collection that don't have a SEG in the segmentation dicom store
• let user load a study, perform segmentation, and push to segmentation dicomstore
• iterate until all studies have been annotated by this annotator

Optional: in the background perform training using annotations available so far and present them to the user with a heatmap of posterior probabilities to guide further annotation.

This scenario is what we have been doing manually in LNQ with nnU-Net and it seems like a powerful paradigm that could be automated.

[fedorov]

@pieper I think this goes beyond the scope of "slicer://", and is more suitable for implementation in mpReview.

FYI, the workflow we have been pitching to Rudolf, you and few other folks with mpReview is summarized here: https://discourse.canceridc.dev/t/annotation-of-data-in-idc-3d-slicer-or-simmilar/505/5, and per feedback of that user, addressed the needs.

[pieper]

@fedorov yes, I know mpReview is a valuable tool, but I'd like to make the case that there is room for examples of other approaches. I think mpReview is still a bit too complex to understand and use in a lot of cases. I've seen the demos several times and it hasn't "clicked" to me that I could easily use it with a clinical colleague for some of my common use cases.

I realize that the "mp" part adds complexity that mpReview needs to handle, so my suggestion is that we consider something less complex, like annotating lesions on studies where a single series is used to segmentation to create training data for a segmentation model. Also mpReview is very Slicer-centric, which may limit its appeal to developers who may want to use a variety of tools in their workflows.

In my experience people are fine with core Slicer tools for segmentation, but keeping track of which studies have been processed and which not is important (many people use dropbox and spreadsheets, which are error-prone). Also being able to make notes or other annotations on a per-study or per-instance basis could be helpful. These notes are useful for QC review and for consultation with other experts for difficult cases. A database like BQ with dashboards and some simple front-end tools could be great for this.

My suggestion would be to have a notebook example that shows how to implement tracking the experiment using SQL and DICOMweb with MONAI or nnU-Net, where the manual segmentation could be Slicer or OHIF or MITK, and they are treated as black-box tools. Such a bare-bones example would be a practical help people (like us) understand and refine how the pieces fit together.

[fedorov]

An interesting scenario could be to make a TODO list for human annotators, something like this:

• for a given collection, segmentation dicomstore, and annotatorID
• query for all and display links for all studies in the collection that don't have a SEG in the segmentation dicom store
• let user load a study, perform segmentation, and push to segmentation dicomstore
• iterate until all studies have been annotated by this annotator

Perhaps mpReview is complex, but it's the simplest approach to implement the above that I can think of right now. With the exception of limiting selection to a specific annotatorID, all of the steps above are already implemented in mpReview.

I realize that the "mp" part adds complexity that mpReview needs to handle, so my suggestion is that we consider something less complex, like annotating lesions on studies where a single series is used to segmentation to create training data for a segmentation model.

It does work with a single series. Did you watch the video in the post I mentioned earlier: https://youtu.be/Oc3_sjqz64s?si=uuJFuA-0V7WgW127&t=104 ?

My suggestion would be to have a notebook example that shows how to implement tracking the experiment using SQL and DICOMweb with MONAI or nnU-Net, where the manual segmentation could be Slicer or OHIF or MITK, and they are treated as black-box tools.

We should probably discuss this, this will be easier. If mpReview is too complex for someone, I cannot see how setting up a custom fork of OHIF Viewer that can be pointed to 2 DICOM stores can be even feasible - and this is just one step that is not straightforward. Also, I think the scope of this ticket goes well beyond the scope of this repository, which was intended to support opening IDC content in Slicer. And this repository and is being deprecated anyway, since all of the URL opening functionality will go into SlicerIDCBrowser.

[pieper]

Yes, we should discuss. Your concept of easy and mine seem very far apart right now but I'm sure we'll reach agreement at some point.

[vkt1414]

What excites me in all of this is the infinite number of things we can think of that could be automated with a convenience of a clickable URL. One of such use cases I constantly stumble upon is to view TotalSegmentator results. While OHIF is great that it could point to two dicom stores, we find OHIF lacking a lot of times. So having that flexibility with the underlying logic is going help a lot.

So, I'm excited for all the use cases resulting as a byproduct of this feature in future. So, thanks Steve for setting the bar we can reach with this feature.

[fedorov]

If we want to support dual DICOM stores, then this becomes a feature for the original Slicer viewer endpoint URL handler, which would need to be able to accept more than one DICOM store - there's not much specific to IDC URL handler here. And as we discussed on multiple occasions earlier, this very very quickly becomes way too complicated, since the user would need to install Google Cloud SDK, which as we experienced before can be difficult even for a CS PhD. I do know Steve talked about other ways that may be feasible to access GCP DICOM stores without Google Cloud SDK, but I have yet to see even a Slicer demo of that.

Let's have the relatively simple feature of opening IDC-specific URLs that can be fetched from public buckets, and discuss other scenarios in a dedicated meeting.