Closed jcfr closed 5 months ago
Proposed commit title & message for Squash & Merge
:
ENH: Add Visual DICOM Browser
Introduces the `ctkDICOMVisualBrowserWidget`, which improves the threaded
execution of the following operations:
- Filtering and navigation with thumbnails of local database and servers results
- Import from file system to local database
- Query/Retrieve from servers (DIMSE C-GET/C-MOVE )
- Storage listener
- Send (emits only a signal for the moment, requires external implementation)
- Remove (only from local database, not from server)
- Metadata exploration
In addition, the commit introduces the following classes:
* `Core` classes:
* `ctkAbstractJob`
* `ctkAbstractScheduler`
* `ctkAbstractWorker`
* `DICOM/Core` classes:
* `ctkDICOMEcho`
* `ctkDICOMInserter`
* `ctkDICOMInserterJob`
* `ctkDICOMInserterWorker`
* `ctkDICOMJob`
* `ctkDICOMJobResponseSet`
* `ctkDICOMQueryJob`
* `ctkDICOMQueryWorker`
* `ctkDICOMRetrieveJob`
* `ctkDICOMRetrieveWorker`
* `ctkDICOMScheduler`
* `ctkDICOMServer`
* `ctkDICOMStorageListener`
* `ctkDICOMStorageListenerJob`
* `ctkDICOMStorageListenerWorker`
* `DICOM/Widget`classes:
* `ctkDICOMPatientItemWidget`
* `ctkDICOMSeriesItemWidget`
* `ctkDICOMServerNodeWidget2`
* `ctkDICOMStudyItemWidget`
Co-authored-by: Andras Lasso <lasso@queensu.ca>
Co-authored-by: Jean-Christophe Fillion-Robin <jchris.fillionr@kitware.com>
cc: @lassoan @Punzo
@jcfr thanks for all the rebasing and fixes!!!! just few notes for the PR messagges:
1) this is the updated UML diagram:
2) and here the updated scripts:
import os
server = ctk.ctkDICOMServer()
server.connectionName = "ConnectionName"
server.callingAETitle = "callingAETitle"
server.calledAETitle = "calledAETitle"
server.host = "host"
server.port = 11112
server.retrieveProtocol = ctk.ctkDICOMServer.CGET
DICOMDatabase = qt.QSettings().value(slicer.dicomDatabaseDirectorySettingsKey)
if not os.path.isdir(DICOMDatabase):
os.makedirs(DICOMDatabase)
browser = ctk.ctkDICOMVisualBrowserWidget()
browser.addServer(server)
browser.setDatabaseDirectory(DICOMDatabase)
browser.filteringPatientID = 1
browser.resize(slicer.util.mainWindow().size)
browser.show()
browser.onQueryPatients()
server = ctk.ctkDICOMServer()
server.connectionName = "ConnectionName"
server.callingAETitle = "callingAETitle"
server.calledAETitle = "calledAETitle"
server.host = "host"
server.port = 11112
server.retrieveProtocol = ctk.ctkDICOMServer.CGET
scheduler = ctk.ctkDICOMScheduler()
scheduler.setDicomDatabase(slicer.dicomDatabase)
scheduler.addServer(server)
nDays = 30
endDate = qt.QDate().currentDate()
startDate = endDate.addDays(-nDays);
parameters = {
"ID": "CT",
#"Name": "name",
#"Study": "description",
#"Series": "description",
#"Modalities": ["CT", "MR"],
#"StartDate": startDate.toString("yyyyMMdd"),
#"EndDate": endDate.toString("yyyyMMdd")
}
scheduler.setFilters(parameters)
# Use one of the following methods: all these methods run background processes!!!!
#scheduler.queryPatients()
#scheduler.queryStudies('patientID')
#scheduler.querySeries('patientID', 'studyInstanceUID')
#scheduler.queryInstances('patientID', 'studyInstanceUID', 'seriesInstanceUID')
#scheduler.retrieveStudy('patientID', 'studyInstanceUID')
#scheduler.retrieveSeries('patientID','studyInstanceUID', 'seriesInstanceUID')
#scheduler.retrieveSOPInstance('patientID', 'studyInstanceUID', 'seriesInstanceUID', 'SOPInstanceUID')
Some of the tests are failing ... it would be great to fix these as well:
$ ctest -R ctkDI
Test project /home/jcfr/Projects/CTK-Qt5-build/CTK-build
Start 182: ctkDICOMCoreTest1
1/47 Test #182: ctkDICOMCoreTest1 .................... Passed 0.04 sec
[...]
Start 315: ctkDICOMApplicationTest1
47/47 Test #315: ctkDICOMApplicationTest1 ............. Passed 0.23 sec
81% tests passed, 9 tests failed out of 47
[...]
Total Test time (real) = 32.83 sec
The following tests FAILED:
195 - ctkDICOMEchoTest1 (Failed)
199 - ctkDICOMQueryTest2 (Failed)
201 - ctkDICOMRetrieveTest2 (Failed)
202 - ctkDICOMSchedulerTest1 (Failed)
203 - ctkDICOMTesterTest1 (Failed)
204 - ctkDICOMTesterTest2 (Failed)
216 - ctkDICOMAppWidgetTest1 (SEGFAULT)
217 - ctkDICOMBrowserTest (Subprocess aborted)
305 - ctkDICOMHostTest1 (Failed)
Some of the tests are failing ... it would be great to fix these as well:
$ ctest -R ctkDI Test project /home/jcfr/Projects/CTK-Qt5-build/CTK-build Start 182: ctkDICOMCoreTest1 1/47 Test #182: ctkDICOMCoreTest1 .................... Passed 0.04 sec [...] Start 315: ctkDICOMApplicationTest1 47/47 Test #315: ctkDICOMApplicationTest1 ............. Passed 0.23 sec 81% tests passed, 9 tests failed out of 47 [...] Total Test time (real) = 32.83 sec The following tests FAILED: 195 - ctkDICOMEchoTest1 (Failed) 199 - ctkDICOMQueryTest2 (Failed) 201 - ctkDICOMRetrieveTest2 (Failed) 202 - ctkDICOMSchedulerTest1 (Failed) 203 - ctkDICOMTesterTest1 (Failed) 204 - ctkDICOMTesterTest2 (Failed) 216 - ctkDICOMAppWidgetTest1 (SEGFAULT) 217 - ctkDICOMBrowserTest (Subprocess aborted) 305 - ctkDICOMHostTest1 (Failed)
ok I am running them and double-checking.
It may also be related to the second part of https://github.com/commontk/CTK/pull/1142#issuecomment-1856174973. Reporting here:
Note/question for @jcfr: To run the test locally, I had to follow these steps:
CTK_BUILD_ALL OFF
CTK_BUILD_ALL_APPS OFF
CTK_BUILD_ALL_LIBRARIES OFF
CTK_BUILD_ALL_PLUGINS OFF
CTK_BUILD_EXAMPLES ON
CTK_ENABLE_DICOM ON
CTK_ENABLE_DICOMApplicationHos OFF
CTK_ENABLE_PluginFramework OFF
CTK_ENABLE_Python_Wrapping OFF
CTK_ENABLE_Widgets ON
Ran make -j 20 in CTK-Build
Manually ran make install in CTK-Build to generate DCMTK binaries in CTK-Build/CMakeExternals/Install/bin
Added CTK-Build/CMakeExternals/Install/lib to my LD_LIBRARY_PATH environment variable
Then I could run the test in CTK-Build/CTK-build folder (ctest)
I might be missing something, but it would be helpful if the superbuild could automatically install DCMTK (eliminating the need for steps 3 and 4). Please let me know if this is a bug or if I am overlooking something.
@jcfr I confirm with the ld library path fixed manually, only these tests fails:
he following tests FAILED: 167 - ctkDICOMSchedulerTest1 (Failed) 181 - ctkDICOMAppWidgetTest1 (SEGFAULT) 182 - ctkDICOMBrowserTest (Subprocess aborted)
I am going to fix them now.
If you can have a look how to fix the installation of DCMTK, it would be great (I don't want to mess up the cmake for projects relying on CTK).
re: LD_LIBRARY_PATH + lookup binaries
Thanks for the details. I will look into fixing the build-system to automate this.
re: UML & testing scripts
I updated the main description based of https://github.com/commontk/CTK/pull/1165#issuecomment-1888809913 to include the updated diagram & testing scripts
@jcfr tests fixed, I could not push on your branch, I opened this PR https://github.com/jcfr/CTK/pull/1
Thanks @Punzo :pray: , your changes were helpful to move forward.
Fixes and improvement to the DICOM tests are being integrated in the following pull requests:
Once there are integrated, we will rebase this one and ensure all tests are still passing :100: :rocket:
Thanks for your patience :hourglass_flowing_sand:
Thank you @jcfr for workong on this a lot! What is the status of this now? Can I help with anything to get this integrated?
Proposed plan:
Sounds good, thank you!
@jcfr can you please add this commit https://github.com/jcfr/CTK/pull/2 before merging? thanks in advance!
The API has been cleanup, @Punzo could you also test on your side ?
Summary:
@Punzo What do you think of updating one of the test by adding a QSignalSpy and checking that these new signals are effectively invoked when removing data ?
@jcfr thanks for all the cleaning! I will test right now.
@Punzo What do you think of updating one of the test by adding a QSignalSpy and checking that these new signals are effectively invoked when removing data ?
ok, I will have a look.
@jcfr thanks for all the cleaning! I will test right now.
@jcfr found a small bug while testing the ctkDICOMVisualBrowser binaries (the GUI was not updating), see PR at https://github.com/jcfr/CTK/pull/3
testing now the PR with Slicer.
testing now the PR with Slicer.
Done, everything working (with the small patch https://github.com/jcfr/CTK/pull/3)!
@Punzo What do you think of updating one of the test by adding a QSignalSpy and checking that these new signals are effectively invoked when removing data ?
ok, I will have a look.
Done, see https://github.com/jcfr/CTK/pull/3/commits/923eaeb282112b7410a0cdeb4a17c690597f6ded
I am in the process of finalizing the integration of some dependent DICOM tests fix, I will then rebase this PR and integrate it as well.
Thanks for the patience
@jcfr would be possible to merge this, please?
I am currently working on the job list UI (https://github.com/commontk/CTK/pull/1184) and I would like to avoid conflicts and multiple rebasing.
If you think further cleaning is necessary, no problem. I will rebase 1184 once you merge into master.
Are the commits going to be squashed? I don't think we need to separate commits for adding new code and then for making fixes or enhancements on that new code - it could be all just one commit to keep the version history less noisy.
Are the commits going to be squashed? I don't think we need to separate commits for adding new code and then for making fixes or enhancements on that new code - it could be all just one commit to keep the version history less noisy.
I think Jc was planning to squash before mergning with this commit message https://github.com/commontk/CTK/pull/1165#issuecomment-1888661303
I am finalizing the cleanups and Will get this integrated this afternoon.
I am finalizing the cleanups and Will get this integrated this afternoon.
ok thanks a lot!
@Punzo Could you perform a last round of test ?
Simplify copy of JobResponseSet introducing clone()
(one of the most recent)ctkAbstractScheduler
was renamed to ctkJobScheduler
as it is not abstract anymore. You may want to update the diagramctkDICOMWorker
but is not fulfilling any purpose, we could keep it around ...ctest -R DICOM -C <buildType> -j10
)@Punzo Could you perform a last round of test ?
sure! done and everything looks good. Thanks a lot for your hard work on the cleaning, I appreciate it.
- Consider reviewing the commit
Simplify copy of JobResponseSet introducing clone()
(one of the most recent)
Checked, looks good to me. I have also tested with valgrind/massif and the memory usage/deallocation works as expected.
- Note that
ctkAbstractScheduler
was renamed toctkJobScheduler
as it is not abstract anymore. You may want to update the diagram- While the class
ctkDICOMWorker
but is not fulfilling any purpose, we could keep it around ...
I agree, I removed it in https://github.com/commontk/CTK/pull/1184/commits/6afcfe5cb9772c3f950bf704334d7f0558d11ad6
and here the new UML
- Thanks to Fix parallel execution of DICOM tests #1182 and Simplify DICOM testing removing needs for installing CTK #1167, DICOM tests are now expected to run in few a seconds and in parallel (you could use
ctest -R DICOM -C <buildType> -j10
)
ah that's great!!!
🍾 Amazing job guys, really looking forward to seeing this working in Slicer!
:warning: Changes originally introduced in commit 8a0717da81ff32cea34a724cfac93a443bbe3caa via this pull request have been removed from the
main
branch using force push. :warning::arrow_right: These changes have been replaced by the commit 88ff72b9.
Rationale:
After performing a
Squash & Merge
operation in association with this pull request (#1165(, the primary authorship of the commit was inadvertently attributed to @jcfr instead of @Punzo, who contributed significantly to the changes.To rectify this, a force push was applied to explicitly set @Punzo as the main author of the commit, reflecting their substantial contribution to the work.
This pull request introduces the
ctkDICOMVisualBrowserWidget
, which improves the threaded execution of various operations:Key Components:
Core
classes:ctkAbstractJob
ctkAbstractScheduler
ctkAbstractWorker
DICOM/Core
classes:ctkDICOMEcho
ctkDICOMInserter
ctkDICOMInserterJob
ctkDICOMInserterWorker
ctkDICOMJob
ctkDICOMJobResponseSet
ctkDICOMQueryJob
ctkDICOMQueryWorker
ctkDICOMRetrieveJob
ctkDICOMRetrieveWorker
ctkDICOMScheduler
ctkDICOMServer
ctkDICOMStorageListener
ctkDICOMStorageListenerJob
ctkDICOMStorageListenerWorker
DICOM/Widget
classes:ctkDICOMPatientItemWidget
ctkDICOMSeriesItemWidget
ctkDICOMServerNodeWidget2
ctkDICOMStudyItemWidget
Related
Testing
Additionally, a CTK application named
ctkDICOMVisualBrowser
has been included for testing purposes (requires enabling CMake optionsexamples
andDICOM
).Testing can also be performed by running the visual DICOM browser from the Slicer Python console:
Similarly, here is how to test the scheduler from the Python console (e.g. query and retrieve):
Video:
https://github.com/commontk/CTK/assets/7985338/a0e362e7-858a-4e47-9298-8d00ae16a23b