search

SMI / SmiServices

Scale-able loading, linking and anonymisation of DICOM images for healthcare research environments (e.g. Safe Havens)
GNU General Public License v3.0
21 stars 5 forks source link

CohortPackager crash when generating reports #1066

Open rkm opened 2 years ago

rkm commented 2 years ago 
2022-02-21 13:37:52.9137|DEBUG|MongoExtractJobStore|Marked job fa8d0d63-f4c5-41d5-aa53-11930f758b43 as completed||
2022-02-21 13:37:52.9174| INFO|FileReporter|Creating report(s) for fa8d0d63-f4c5-41d5-aa53-11930f758b43||
2022-02-21 13:37:52.9527| INFO|FileReporter|Writing reports to directory XXX/projects/PROJ/extractions/reports/XXX||
2022-02-21 13:37:53.1164| INFO|Microservices.CohortPackager.Execution.JobProcessing.ExtractJobWatcher|Stopping (ProcessJob threw an unhandled exception)||
2022-02-21 13:37:53.1164|DEBUG|Microservices.CohortPackager.Execution.JobProcessing.ExtractJobWatcher|Lock released, no more jobs will be processed||
2022-02-21 13:37:53.1174|FATAL|CohortPackagerHost|ExtractJobWatcher threw an exception|System.ArgumentException: expectedAnonFile
   at Microservices.CohortPackager.Execution.ExtractJobStorage.FileAnonFailureInfo..ctor(String expectedAnonFile, String reason) in /home/vsts/work/1/s/src/microservices/Microservices.Cohort
Packager/Execution/ExtractJobStorage/FileAnonFailureInfo.cs:line 28
   at Microservices.CohortPackager.Execution.ExtractJobStorage.MongoDB.MongoExtractJobStore.<>c.<GetCompletedJobAnonymisationFailuresImpl>b__21_3(Tuple`2 x) in /home/vsts/work/1/s/src/micros
ervices/Microservices.CohortPackager/Execution/ExtractJobStorage/MongoDB/MongoExtractJobStore.cs:line 341
   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
   at Microservices.CohortPackager.Execution.JobProcessing.Reporting.JobReporterBase.WriteSplitReport(CompletedExtractJobInfo jobInfo) in /home/vsts/work/1/s/src/microservices/Microservices.
CohortPackager/Execution/JobProcessing/Reporting/JobReporterBase.cs:line 182
   at Microservices.CohortPackager.Execution.JobProcessing.Reporting.JobReporterBase.CreateReport(Guid jobId) in /home/vsts/work/1/s/src/microservices/Microservices.CohortPackager/Execution/
JobProcessing/Reporting/JobReporterBase.cs:line 75
   at Microservices.CohortPackager.Execution.JobProcessing.ExtractJobWatcher.DoJobCompletionTasks(ExtractJobInfo jobInfo) in /home/vsts/work/1/s/src/microservices/Microservices.CohortPackage
r/Execution/JobProcessing/ExtractJobWatcher.cs:line 128
   at Microservices.CohortPackager.Execution.JobProcessing.ExtractJobWatcher.ProcessJobs(Guid specificJob) in /home/vsts/work/1/s/src/microservices/Microservices.CohortPackager/Execution/Job
Processing/ExtractJobWatcher.cs:line 90|
2022-02-21 13:37:53.3668| INFO|Microservices.CohortPackager.Execution.JobProcessing.ExtractJobWatcher|Stopping (Host - Fatal error in MicroserviceHost (ExtractJobWatcher threw an exception))
||
2022-02-21 13:37:53.3668|DEBUG|Microservices.CohortPackager.Execution.JobProcessing.ExtractJobWatcher|Lock released, no more jobs will be processed||
202
rkm commented 2 years ago

The issue is that for status messages originating from an anonymiser which has failed to process a file, the outputFileName will not be set, and instead we need to reference the original file.

Note that theses statuses can indicate an issue with the software including stack traces etc., and may not be appropriate for the final reports which are intended for analyst review.

Note: Another related bug is that CohortPackager won't retry the report generation if there is a crash from a previous run, but the --recreate-report option can be used from the CLI.