Missing studies when using google integration

[fedorov]

Bug Report

Study browser interfacing google healthcare DICOM is not showing all of the studies that are available in a datastore.

Describe the Bug

I do not know how widespread this problem is, but ...

If I use this specific instantiation of OHIF interfacing google DICOM datastore qin-headneck, only 2 studies are shown for PatientID QIN-HEADNECK-01-0003: https://idc-sandbox-000.firebaseapp.com/projects/chc-tcia/locations/us-central1/datasets/qin-headneck/dicomStores/qin-headneck.

I know that patient has 4 studies.

Interestingly, if I manually specify StudyInstanceUID for a missing study, it does load:

https://idc-sandbox-000.firebaseapp.com/projects/chc-tcia/locations/us-central1/datasets/qin-headneck/dicomStores/qin-headneck/study/1.3.6.1.4.1.14519.5.2.1.2744.7002.150059977302243314164020079415

What steps can we follow to reproduce the bug?

You will need access to the IDC project idc-sandbox-000, or you would need to set up another google integration. The dataset is in the public TCIA offering of Google Healthcare. Navigate to datastore qin-headneck, the subject in question is at the top of the list.

This was tested with OHIF 3.3.8.

cc: @dzelemba

[dzelemba]

Thanks for the report Andrey! I tried reproducing this, but haven't been able to, here's what I tried

docker pull ohif/viewer:later
docker run --env CLIENT_ID=<id>.apps.googleusercontent.com --publish 3000:80 ohif/viewer:latest

Navigate to http://localhost:3000/projects/chc-tcia/locations/us-central1/datasets/qin-headneck/dicomStores/qin-headneck and enter QIN-HEADNECK-01-0003 in the PatientID field and I see

[dzelemba]

Also I was able to log into your instance at https://idc-sandbox-000.firebaseapp.com/projects/chc-tcia/locations/us-central1/datasets/qin-headneck/dicomStores/qin-headneck (doesn't look like access is restriced?) and I'm seeing the same thing, am I doing something wrong to reproduce?

[pieper]

Hi @dzelemba - the sandbox app is just a statically hosted ohif instance - only the data is access-controlled (makes perfect sense that your account has access to the TCIA collections 😄 )

As for the underlying issue, I believe it is the sort-order and the limit on the qido query. If I enter QIN-HEADNECK-01-0003 and search for it by MRN I get four studies, but if I sort the list by MRN I only see 2. This is because the OHIF studylist doesn't do a QIDO query when the sort order changes. Also since the QIDO call result is not sorted by MRN only a random subset of the collection is in the studylist and available for sorting. I don't think think this is a google issue, just OHIF and the way the standard works.

[swederik]

the way the standard works.

Yeah maybe we should write a change request to add add orderBy to QIDO-RS.

[fedorov]

This sorting issue must be important for clinical PACS users. How do they solve this?

[dannyrb]

Support for options that are outlined in ODATA would be nice: https://www.odata.org/

[dzelemba]

Ah yes.. that is a known issue with DICOMweb and every viewer I've tried so far. It also seems to be a common problem with any paginated UI that supports sorting on columns.

Supporting orderBy in QidoRs isn't actually an easy feature to support on the server. For small datasets it's fine, but once you have millions of rows, orderBy requires full table scans which can't be done in the latency targets we aim for in Qido. That being said, there may be some heuristics we could figure out to get this to work in a smaller set of use cases that are important.

[pieper]

Agreed - most PACS study browsers discourage or prevent you from populating the study list with all entries, and instead make you pick a subset like "within the last week" and then you can sort within the subset. They won't even show you the list until it is small enough to sort on the client.

Date-based subsets make sense for clinical scenarios, but doesn't work so well for research where a lot of the data is old.

Probably a better solution is to provide a faceted search kind of interface, where see histograms or or pie charts of what's in the database and you can drill down from there (like GDC for instance). Unfortunately we can't get those summary statistics via QIDO so we need another API (for google dicomweb I guess that's big query).

[fedorov]

Thank you for the explanation, this helped a lot. I am removing "IDC:candidate" label, since we will not be using this interface for data browsing, and it is not clear if it can be addressed at all.

I think a related issue is how to communicate to a user like myself this peculiarity of the interface - that the page shown and sorted is a random sample from the server.

[dannyrb]

Most implementations I've come across simply maintain a list of studies and their metadata separately from the PACS so it can be queried free from the limitations of DICOMWeb, or they impose the limitations you've described. There are options for strength/full text search that are performant for millions of rows.

If we know the dataset is small enough, we could request it entirely and perform sorting on the client. We could also provide the ability to map search fields to a function that generates the request, and an optional function to map the request's data to the expected table result format (opening the door to non-dicomweb querying).

[sedghi]

@fedorov how about this one

[fedorov]

Indeed, another one not relevant to IDC at this point.