Closed jazb83 closed 4 years ago
I'm not rex but I do have a few questions that immediately come to mind.
@crcrewso
Thank you for looking at the question.
"The first thing that concerns me is why you're getting any dicom export errors at all with these patients. Do you happen to know why they're failing." I looked to the log files of patient 1. I got this error for the 3 missing plans when Exporting from ECLIPSE to my personal computer: "05/26/2020 21:58:18 Error Object Connection Cannot export RT Plan '1.2.246.352.71.5.568872780110.1285816.20170907162709' with an empty Beam Sequence"
However, when exporting to MIM We missed 5 plans. 3 Plans have the error above but the other two don't seem to have an error but they are not exported ( I will add the log files during the day. I have to take some time to anonymize the data consistently for all the exports ).
"Secondly, which version of Aria/Eclipse are you using?" Versions of Software: varian ECLIPSE: External Beam Planning. Version: 15.6 MIM Version 6.9.3 (Build J510-0) evilDicom Version 2.0.6.5
"Is this a tbox?" No, I am working with Clinical Eclipse.
"Is your AE_mim in the Varian DICOM whitelist for the Q/R service you're connecting to?" Yes, but I was not doing the export to MIM. I did it after I read your answer.
Please just give me some time to update the question with all your requested info. Thanks again for your help.
Of course, take all the time you need. If I could make a testing suggestion, could you try one plan at a time until you've found a failure on evilDicom that succeeded in DICOM Import/Export. It should make the logs shorter and the comparisons easier.
You may need to get access to your aria dicom server's D drive to see these logs.
I can tell you that the Varian DICOM export service is DIFFERENT than the Varian DICOM DbDaemon. They are completely different programs. I have seen similar behavior myself (try exporting a uncertainty dose matrix if you want to see the daemon crash). As to your exact problem, I don't have time to really investigate to the level you would need. From your messages, it looks like the failing is on the Varian side and not EvilDICOM anyway.
It is not transferring the RTPLANS because they have some missing information.
For one patient, I tried to transfer 14 RTPLANS but 13 Failed and 1 Succeeded.
For all the RTPLANS that failed, I got this similar message:
-----> ------------Receiving data to my local AET // ?plan = Visual Studio Command to see the value of variable plan ?plan = requested serie {ISO_IR 192|STUDY|VMSDBD|RTPLAN|ARIA RadOnc Plans|1.2.246.352.71.1.XXXXX.305577.XXXXXXXXX|1.2.246.352.71.2.XXXXX.12231533.XXXXXXX0|5} Elements: Count = 8 Modality: "RTPLAN" QueryLevel: STUDY SeriesDescription: "ARIA RadOnc Plans" SeriesInstanceUID: "1.2.246.352.71.2.XXXXXX.12231533.XXXXXXX" SeriesNumber: 5 _sel: {EvilDICOM.Core.Selection.DICOMSelector} ?modImg = requested RTPLAN {ISO_IR 192|1.2.246.352.70.1.70|1.2.246.352.71.5.XXXXXXX.1439821.20190509110317|STUDY|VMSDBD|1.2.246.352.71.2.XXXXXXX.12231533.XXXXXXX|Prostate QA||5/9/2019 12:00:00 AM|6/1/2020 11:03:18 AM|10|UNAPPROVED} ApprovalStatus: "UNAPPROVED" Elements: Count = 12 NumberOfBeams: 10 PlanDate: {5/9/2019 12:00:00 AM} PlanDescription: "" PlanLabel: "Prostate QA" PlanTime: {6/1/2020 11:03:18 AM} QueryLevel: STUDY ReferencedPlan: ("", "", "") SOPClassUID: "1.2.246.352.70.1.70" SOPInstanceUID: "1.2.246.352.71.5.XXXXXXX.1439821.20190509110317" _sel: {EvilDICOM.Core.Selection.DICOMSelector} ?response {CMoveResponse - WARNING 0/0} AffectedSOPClassUID: "1.2.840.10008.5.1.4.1.2.2.2" CommandField: 32801 Data: {DICOM Object [Unknown] : 1 Elements} DataPresentationContextId: 1 DataSetType: 0 Elements: Count = 10 GroupLength: 114 HasData: true MessageIDBeingRespondedTo: 1 NumberOfCompletedOps: 0 NumberOfFailedOps: 1 NumberOfRemainingOps: 0 NumberOfWarningOps: 0 Status: 45056 _affectedSOPClassUID: {(0000,0002) : AffectedSOPClassUID (UniqueIdentifier) -> 1.2.840.10008.5.1.4.1.2.2.2} _commandField: {(0000,0100) : CommandField (UnsignedShort) -> 32801} _data: {DICOM Object [Unknown] : 1 Elements} _dataSetType: {(0000,0800) : CommandDataSetType (UnsignedShort) -> 0} _groupLength: {(0000,0000) : Group header for group 0000 (UnsignedLong) -> 114} _messageIdBeingRespondedTo: {(0000,0120) : MessageIDBeingRespondedTo (UnsignedShort) -> 1} _numCompletedOps: {(0000,1021) : NumberOfCompletedSuboperations (UnsignedShort) -> 0} _numFailedOps: {(0000,1022) : NumberOfFailedSuboperations (UnsignedShort) -> 1} _numRemainingOps: {(0000,1020) : NumberOfRemainingSuboperations (UnsignedShort) -> 0} _numWarningOps: {(0000,1023) : NumberOfWarningSuboperations (UnsignedShort) -> 0} _status: {(0000,0900) : Status (UnsignedShort) -> 45056} <-----
Unfortunately, the status message _status: {(0000,0900) : Status (UnsignedShort) -> 45056} and Data: {DICOM Object [Unknown] : 1 Elements} don't give more information about it.
So, I tried to transfer/retrieve the same RT Plans using DCMTK commands, and I got the following similar messages for all the failed plans:
-----> D: Response Identifiers: D: D: # Dicom-Data-Set D: # Used TransferSyntax: Little Endian Implicit D: (0008,0058) UI [1.2.246.352.71.5.XXXXXXX.1439821.20190509110317] # 52, 1 FailedSOPInstanceUIDList D: D: Status Detail: D: D: # Dicom-Data-Set D: # Used TransferSyntax: Little Endian Implicit D: (0000,0902) LO (no value available) # 0, 0 ErrorComment <----- Now, checking the definition of the tags:
E Command Dictionary (Normative) (0000,0900) | Status | Status | US | 1 | Confirmation status of the operation.
C.5 Failure Status Class. Error Comment | (0000,0902) | LO | 1 | This optional field contains an application-specific text description of the error detected. From this one, we have a bunch of errors but none of them match the (no value available or DICOM Object [Unknown] .
Luckily, I found a post asking about if it was possible to move an instance with an empty Patient ID :
"An SCU sends an instance to a storage SCP and then uses a Patient Root MOVE service to move the instance to a different system. The instance has an empty Patient ID (0010,0020) [tag is present, but value is empty] , which is allowed as PS 3.3 defines Patient ID as a type 2 tag in table C.7-1.
The SCU will send the following C-MOVE request to move the instance:
0008,0052: STUDY
0010,0020:
PS 3.4 defines in C.4.2.2.1 Baseline Behavior of a C-MOVE SCU: The SCU shall supply a single value in the Unique Key Attribute for each level above the Query/Retrieve level.
Should the SCP reject the request because of the empty Patient ID tag (0010,0020) as it is not a single value or should it match and move the study that has an empty Patient ID and the specified Study UID? The current standard is very unclear if an empty value is a valid ‘single value’ or not."
The brief answer was: "This would be illegal (unless relational retrieval has been negotiated, in which case the attribute shouldn't be sent at all, not sent empty), and yes, the SCP should reject the request with an A900 "Identifier does not match SOP Class" error."
Ok, it was not exactly our error message but I tried to look if the failed plans were missing the patID (0010,0020). So, I queried the patID value using the following DCMTK command:
findscu -v -d -aet DCMTK -aec VMSDBD 10.XXXXXXX 51XXX -S -k PatientID=1905XXXX -k QueryRetrieveLevel=IMAGE -k "0020,000D=1.2.840.113619.2.278.3.XXXXXXX.764.XXXXXXX.374" -k "0020,000E=1.2.246.352.71.2.XXXXXXX.11569674.2017XXXXXXX" -k "0008,0018=1.2.246.352.71.5.XXXXXXX.1439821.201XXXXXXX" -k "0010,0020"
and I got:
-----> I: Request Identifiers: I: I: # Dicom-Data-Set I: # Used TransferSyntax: Little Endian Explicit I: (0008,0018) UI [1.2.246.352.71.5.XXXXXXX.1439821.201XXXXXXX] # 52, 1 SOPInstanceUID I: (0008,0052) CS [IMAGE] # 6, 1 QueryRetrieveLevel I: (0010,0020) LO (no value available) # 0, 0 PatientID I: (0020,000d) UI [1.2.840.113619.2.278.3.XXXXXXX.764.XXXXXXX.374] # 52, 1 StudyInstanceUID I: (0020,000e) UI [1.2.246.352.71.2.XXXXXXX.11569674.2017XXXXXXX] # 54, 1 SeriesInstanceUID I: D: DcmDataset::read() TransferSyntax="Little Endian Implicit" I: Received Final Find Response D: ===================== INCOMING DIMSE MESSAGE ==================== D: Message Type : C-FIND RSP D: Message ID Being Responded To : 1 D: Affected SOP Class UID : FINDStudyRootQueryRetrieveInformationModel D: Data Set : none D: DIMSE Status : 0x0000: Success D: ======================= END DIMSE MESSAGE ======================= I: Releasing Association <-----
and effectively, the value is missing for all the RTPLANS that failed, but when I tried the same command with the single RTPLAN that succeeded:
-----> D: Response Identifiers: D: D: # Dicom-Data-Set D: # Used TransferSyntax: Little Endian Implicit D: (0008,0005) CS [ISO_IR 192] # 10, 1 SpecificCharacterSet D: (0008,0018) UI [1.2.246.352.71.5.568XXXXXXXXXX.1298872.XXXXXX] # 52, 1 SOPInstanceUID D: (0008,0052) CS [IMAGE ] # 6, 1 QueryRetrieveLevel D: (0008,0054) AE [VMSDBD] # 6, 1 RetrieveAETitle D: (0010,0020) LO [1905XXXXX] # 8, 1 PatientID I: (0020,000d) UI [1.2.840.113619.2.278.3.XXXXXXX.764.XXXXXXX.374] # 52, 1 StudyInstanceUID I: (0020,000e) UI [1.2.246.352.71.2.XXXXXXX.11569674.2017XXXXXXX] # 54, 1 SeriesInstanceUID D: D: DcmDataset::read() TransferSyntax="Little Endian Implicit" I: Received Final Find Response D: ===================== INCOMING DIMSE MESSAGE ==================== D: Message Type : C-FIND RSP D: Message ID Being Responded To : 1 D: Affected SOP Class UID : FINDStudyRootQueryRetrieveInformationModel D: Data Set : none D: DIMSE Status : 0x0000: Success D: ======================= END DIMSE MESSAGE ======================= I: Releasing Association <-----
The patient ID is not missing and this RTPLAN was retrieved successfully.
I tested with other patients and I got similar errors: "(0000,0902) LO (no value available) # 0, 0 ErrorComment". However, some patients do have the patID present but still the transfering/moving failed.
Then I believe that if some attribute/value/something important is missing, it is not possible to retrieve/transfer the data. It is I guess similar to what @rexcardan describe above: "(try exporting a uncertainty dose matrix if you want to see the daemon crash)".
Thanks for your help @crcrewso and @rexcardan.
Wait? You're missing PID? um, To my understanding that breaks DICOM and is completely against the standard. No wonder you're having issues. Across our clinic (and most clinics I'm aware of) there is a format for not-real patients. I wasn't aware Aria could even handle missing PID's safely.
For a multitude of reasons I would strongly recommend you find all patients with missing PID and add one in whichever format your clinic finds acceptable.
Well, sometimes PID is missing sometimes other things are missing. It is just crazy!!
Thanks for the recommendation.
Versions of Software: varian ECLIPSE: External Beam Planning. Version: 15.6 MIM Version 6.9.3 (Build J510-0) evilDicom Version 2.0.6.5
I am using the following script to retrieve RTPLANS for some patients:
I managed to retrieve (pull) the RTPLANS to my computer for some patients.
Example Patient 1: We get almost all plans with evilDicom
Console Output: Sending serie number of plan 1.2.840.113619.2.55.3.XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 7 Number of Failed Operations : 0 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.246.352.71.2.XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 2 Number of Failed Operations : 0 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.246.352.71.2.XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 7 Number of Failed Operations : 5 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.246.352.71.2..XXXXXX.XXXXX.... Sending serie number of plan 1.2.246.352.71.2..XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 9 Number of Failed Operations : 0 Number of Remaining Operation:0 Number of Warning Operations : 0 and my output folder looks like this:
and the output folder gets the RTPLANS:
In total, I got 26 RTPLANS.
TODO: Try to add log files with errors of the plans that failed (if possible)
I went to eclipse and use the conventional export dicom method to test that I got the right number of plans for patient 1 using evilDicom.
1) Select export dicom
2) authorize plans (if needed)
3) Export
4) The errors from the log file for the plans that were not exported:
05/26/2020 21:58:18 Error Object Connection Cannot export RT Plan '1.2.246.352.71.5.568872780110.1285816.20170907162709' with an empty Beam Sequence. Please check the number and type of beams in the plan and export configuration option. 05/26/2020 21:58:18 Error Object Connection DICOM conversion failed 05/26/2020 21:58:18 Plan: (1.2.246.352.71.5.568872780110.1285816.20170907162709) Error Object Conversion Object conversion failed with 2 messages. 05/26/2020 21:58:18 Plan: (1.2.246.352.71.5.568872780110.1285816.20170907162709) Error Object Conversion Object conversion failed. Message 1/2: Cannot export RT Plan '1.2.246.352.71.5.568872780110.1285816.20170907162709' with an empty Beam Sequence. Please check the number and type of beams in the plan and export configuration option. 05/26/2020 21:58:18 Plan: (1.2.246.352.71.5.568872780110.1285816.20170907162709) Error Object Conversion Object conversion failed. Message 2/2: DICOM conversion failed 05/26/2020 21:58:18 Info Object Connection Exporting RT Plan '20170907_0435' (1.2.246.352.71.5.568872780110.1285841.20170907163535) 05/26/2020 21:58:18 Info Object Connection Fractionation '20170907_0435' 05/26/2020 21:58:18 Info Object Connection Loading scheduled images... 05/26/2020 21:58:18 Info Object Connection No active session found for Course 'Auto_KBP_Hal' 05/26/2020 21:58:18 Info Object Connection Done loading scheduled images. 05/26/2020 21:58:18 Error Object Connection Cannot export RT Plan '1.2.246.352.71.5.568872780110.1285841.20170907163535' with an empty Beam Sequence. Please check the number and type of beams in the plan and export configuration option. 05/26/2020 21:58:18 Error Object Connection DICOM conversion failed 05/26/2020 21:58:18 Plan: (1.2.246.352.71.5.568872780110.1285841.20170907163535) Error Object Conversion Object conversion failed with 2 messages. 05/26/2020 21:58:18 Plan: (1.2.246.352.71.5.568872780110.1285841.20170907163535) Error Object Conversion Object conversion failed. Message 1/2: Cannot export RT Plan '1.2.246.352.71.5.568872780110.1285841.20170907163535' with an empty Beam Sequence. Please check the number and type of beams in the plan and export configuration option. 05/26/2020 21:58:18 Plan: (1.2.246.352.71.5.568872780110.1285841.20170907163535) Error Object Conversion Object conversion failed. Message 2/2: DICOM conversion failed 05/26/2020 21:58:18 Info Object Connection Exporting RT Plan '20170907_0441' (1.2.246.352.71.5.568872780110.1285861.20170907164115) 05/26/2020 21:58:18 Info Object Connection Fractionation '20170907_0441' 05/26/2020 21:58:18 Info Object Connection Loading scheduled images... 05/26/2020 21:58:18 Info Object Connection No active session found for Course 'Auto_KBP_Hal' 05/26/2020 21:58:18 Info Object Connection Done loading scheduled images. 05/26/2020 21:58:18 Error Object Connection Cannot export RT Plan '1.2.246.352.71.5.568872780110.1285861.20170907164115' with an empty Beam Sequence. Please check the number and type of beams in the plan and export configuration option. 05/26/2020 21:58:18 Error Object Connection DICOM conversion failed 05/26/2020 21:58:18 Plan: (1.2.246.352.71.5.568872780110.1285861.20170907164115) Error Object Conversion Object conversion failed with 2 messages. 05/26/2020 21:58:18 Plan: (1.2.246.352.71.5.568872780110.1285861.20170907164115) Error Object Conversion Object conversion failed. Message 1/2: Cannot export RT Plan '1.2.246.352.71.5.568872780110.1285861.20170907164115' with an empty Beam Sequence. Please check the number and type of beams in the plan and export configuration option. 05/26/2020 21:58:18 Plan: (1.2.246.352.71.5.568872780110.1285861.20170907164115) Error Object Conversion Object conversion failed. Message 2/2: DICOM conversion failed
TODO: You will have to anonymize consistently the logFiles for evilDicom, EclipseToLocalComputer and EclipseToMim.
link to log file: https://www.dropbox.com/sh/fv6tw0fd3fuozls/AABe5mydnBa2k_ZbvBoyIq6pa?dl=0
Exporting to MIM
1) Authorize plan (if needed)
2) Export
3) Double-check export data in MIM
4) TODO: Errors from log file
TODO: Link to Log Files (anonymize data)
We got 25 Plans exporting from ECLIPSE to MIM
Balance:
evilDicom exporting to my personal computer: 26 ECLIPSE exporting to my personal computer: 27 ECLIPSE exporting to MIM : 25
I can still work with that because I got the plans that I need.
TODO: export patient 2 and 3 to MIM for comparison TODO: Add results from logFile TODO: All examples must follow the same format
Now, trying both methods with patient 2: We got a single plan using evilDicom
evilDicom 1 vs ECLIPSE 14 plans
evilDicom output:
Sending serie number of plan 1.2.246.352.71.2.187559867970..XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 0 Number of Failed Operations : 3 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.246.352.71.2.XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 0 Number of Failed Operations : 1 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.246.352.71.2.XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 1 Number of Failed Operations : 5 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.840.113619.6.196..XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 0 Number of Failed Operations : 2 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.246.352.71.2.568872780110..XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 0 Number of Failed Operations : 1 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.246.352.71.2..XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 0 Number of Failed Operations : 0 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.246.352.71.2..XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 0 Number of Failed Operations : 1 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.246.352.71.2..XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 0 Number of Failed Operations : 0 Number of Remaining Operation:0 Number of Warning Operations : 0
Eclipse output:
Patient 3: We got 0 plan using evilDicom evilDicom 0 vs Eclipse 9
evilDicom console output: Sending serie number of plan 1.2.246.352.71.2.568872780110..XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 0 Number of Failed Operations : 2 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.840.113619.6.196..XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 0 Number of Failed Operations : 0 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.246.352.71.2.568872780110..XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 0 Number of Failed Operations : 5 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.246.352.71.2..XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 0 Number of Failed Operations : 1 Number of Remaining Operation:0 Number of Warning Operations : 0 Sending serie number of plan 1.2.246.352.71.2.568872780110..XXXXXX.XXXXX.... DICOM C- Move Results : Number of Completed Operations : 0 Number of Failed Operations : 1 Number of Remaining Operation:0 Number of Warning Operations : 0
Eclipse Output:
A similar path follows for other patients.
Questions: 1) What type/kind of RTPLANS can be retrieved using evilDicom? 2) What are the constraints/restrictions?
Note: -I can retrieve (pull) approved and unapproved plans using evilDicom.
https://www.dropbox.com/s/pk5t0ats40me8o2/evilfailed-RP.1.2.246.352.71.5.187559867970.1497100.20190509110519.txt?dl=0