Closed drtjre closed 8 years ago
To permit others to test this situation, I have made available on USBshare three cases in anonymous form. Below a description of how these can be tested to replicate this bug.
Three Anonymized Stolz cases are privately (requires login) available on USBshare/StolzAnonymize.
Manually performing LungSegmentation/Quant using Fiji produces legitimate results:
PatientName (Anonymized) StudyID LungVol PD15 LAA-950mm^3
Anonymous Sid 10007 SID10007 2.841 -858 49240.431
Anonymous Sid 10113 SID10113 3.68 -886 41568.794
Anonymous Sid 10201 SID10201 4.471 -937 390522.385
The 3 anonymized cases have been loaded into Osirix for testing with RIQAE - see below for results.
CREATE TEMPORARY TABLE stolz3pacs USING fourquant.pacs OPTIONS (bind "YETI", username "OSIRIX", server "localhost", port "11112", path "StolzAnonymous3.csv");
where StolzAnonymous3.csv is:
Patient Name;Patient ID;Study Instance UID;Study Date;Study Description;Series Instance UID;Series Description;Result
Anonymous^Sid^10007;;;;;;;
Anonymous^Sid^10113;;;;;;;
Anonymous^Sid^10201;;;;;;;
0: jdbc:hive2://127.0.0.1:8000/> select PatientName from stolz3pacs;
+----------------------+--+
| PatientName |
+----------------------+--+
| Anonymous Sid 10007 |
| Anonymous Sid 10201 |
| Anonymous Sid 10113 |
+----------------------+--+
This appears to produce non-zero reasonable results as expected:
select patientname,seriesDescription,runmap(fetch_dicom_imagej(studyInstanceUID,seriesInstanceUID),"Measure","") from stolz3pacs where success=true;
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.49001.30000008123113283818700000454 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081231.103551.64 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs7121845210281147784.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.54504.30000008112007153707800007751 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081120.125332.60 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs3246684587818035570.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.50021.30000008051505383531200008814 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20080515.113315.4 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs7126723019959534364.properties --directory emptyDir
+----------------------+------------------------+--------------------------------------------------+--+
| patientname | seriesDescription | _c2 |
+----------------------+------------------------+--------------------------------------------------+--+
| Anonymous Sid 10007 | HWS+Thx art. 3.0 B30f | {"Area":[262144.0],"Mean":[215.31087112426758]} |
| Anonymous Sid 10201 | Thorax nativ 3.0 B30f | {"Area":[262144.0],"Mean":[471.34604263305664]} |
| Anonymous Sid 10113 | LE 3.0 B30f | {"Area":[262144.0],"Mean":[762.6416549682617]} |
+----------------------+------------------------+--------------------------------------------------+--+
Trial on "unsegmented" image data results in all values as Zero/null:
select patientname,seriesDescription,studydate,runmap(fetch_dicom_imagej(studyInstanceUID,seriesInstanceUID),"PD LAA Quantify SegmentedLung","") from stolz3pacs where success=true;
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.49001.30000008123113283818700000454 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081231.103551.64 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs6227654999163457109.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.54504.30000008112007153707800007751 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081120.125332.60 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs4131042739232563902.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.50021.30000008051505383531200008814 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20080515.113315.4 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs8837834746381452551.properties --directory emptyDir
+----------------------+------------------------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| patientname | seriesDescription | studydate | _c3 |
+----------------------+------------------------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| Anonymous Sid 10007 | HWS+Thx art. 3.0 B30f | 20081231 | {"FOV voxels ":[8.2313216E7],"LAA-1024%":[0.0],"LAA-1024pixel^3":[0.0],"LAA-900%":[0.0],"LAA-900pixel^3":[0.0],"LAA-950%":[0.0],"LAA-950pixel^3":[0.0],"LungVol pixel^3":[0.0],"LungVol voxs":[0.0],"PD15":[0.0],"StudyID":[NaN],"sex":[NaN],"vox volume pixel^3":[1.0]} |
| Anonymous Sid 10201 | Thorax nativ 3.0 B30f | 20081120 | {"FOV voxels ":[4.6923776E7],"LAA-1024%":[0.0],"LAA-1024pixel^3":[0.0],"LAA-900%":[0.0],"LAA-900pixel^3":[0.0],"LAA-950%":[0.0],"LAA-950pixel^3":[0.0],"LungVol pixel^3":[0.0],"LungVol voxs":[0.0],"PD15":[0.0],"StudyID":[NaN],"sex":[NaN],"vox volume pixel^3":[1.0]} |
| Anonymous Sid 10113 | LE 3.0 B30f | 20080515 | {"FOV voxels ":[5.7933824E7],"LAA-1024%":[0.0],"LAA-1024pixel^3":[0.0],"LAA-900%":[0.0],"LAA-900pixel^3":[0.0],"LAA-950%":[0.0],"LAA-950pixel^3":[0.0],"LungVol pixel^3":[0.0],"LungVol voxs":[0.0],"PD15":[0.0],"StudyID":[NaN],"sex":[NaN],"vox volume pixel^3":[1.0]} |
+----------------------+------------------------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
3 rows selected (17.768 seconds)
A trial which incorporates Image Segmentation also results in zero/null results:
select patientname,seriesDescription,studydate,runmap(run2(fetch_dicom_imagej(studyInstanceUID,seriesInstanceUID),"Segment Lung Tissue",""),"PD LAA Quantify SegmentedLung","") from stolz3pacs where success=true;
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.54504.30000008112007153707800007751 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081120.125332.60 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs6194165006507264746.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.49001.30000008123113283818700000454 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081231.103551.64 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs7131212569564495806.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.50021.30000008051505383531200008814 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20080515.113315.4 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs7668734660244417243.properties --directory emptyDir
+----------------------+------------------------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| patientname | seriesDescription | studydate | _c3 |
+----------------------+------------------------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| Anonymous Sid 10007 | HWS+Thx art. 3.0 B30f | 20081231 | {"FOV voxels ":[8.2313216E7],"LAA-1024%":[0.0],"LAA-1024pixel^3":[0.0],"LAA-900%":[0.0],"LAA-900pixel^3":[0.0],"LAA-950%":[0.0],"LAA-950pixel^3":[0.0],"LungVol pixel^3":[0.0],"LungVol voxs":[0.0],"PD15":[0.0],"StudyID":[NaN],"sex":[NaN],"vox volume pixel^3":[1.0]} |
| Anonymous Sid 10201 | Thorax nativ 3.0 B30f | 20081120 | {"FOV voxels ":[4.6923776E7],"LAA-1024%":[0.0],"LAA-1024pixel^3":[0.0],"LAA-900%":[0.0],"LAA-900pixel^3":[0.0],"LAA-950%":[0.0],"LAA-950pixel^3":[0.0],"LungVol pixel^3":[0.0],"LungVol voxs":[0.0],"PD15":[0.0],"StudyID":[NaN],"sex":[NaN],"vox volume pixel^3":[1.0]} |
| Anonymous Sid 10113 | LE 3.0 B30f | 20080515 | {"FOV voxels ":[5.7933824E7],"LAA-1024%":[0.0],"LAA-1024pixel^3":[0.0],"LAA-900%":[0.0],"LAA-900pixel^3":[0.0],"LAA-950%":[0.0],"LAA-950pixel^3":[0.0],"LungVol pixel^3":[0.0],"LungVol voxs":[0.0],"PD15":[0.0],"StudyID":[NaN],"sex":[NaN],"vox volume pixel^3":[1.0]} |
+----------------------+------------------------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
Note: All values are Null.
My main suspicious is that the offset is somehow being lost, so try running stats
(stats(fetch_dicom_imagej(studyInstanceUID,seriesInstanceUID))
) to get the information. The FOV voxels
still comes out correct which I is strange.
Can you try using runtable
and runrow
instead of runmap
since the result might for some reason be in another row?
Runrow - gives zero. RunTABLE gives exception.
select patientname,seriesDescription,studydate,runrow(fetch_dicom_imagej(studyInstanceUID,seriesInstanceUID),"PD LAA Quantify SegmentedLung","") from stolz3pacs where success=true;
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.54504.30000008112007153707800007751 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081120.125332.60 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs1399583431030434596.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.49001.30000008123113283818700000454 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081231.103551.64 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs5599936374050646508.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.50021.30000008051505383531200008814 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20080515.113315.4 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs5668133840403478511.properties --directory emptyDir
+----------------------+------------------------+------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| patientname | seriesDescription | studydate | _c3 |
+----------------------+------------------------+------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| Anonymous Sid 10007 | HWS+Thx art. 3.0 B30f | 20081231 | {"FOV voxels ":8.2313216E7,"LAA-1024%":0.0,"LAA-1024pixel^3":0.0,"LAA-900%":0.0,"LAA-900pixel^3":0.0,"LAA-950%":0.0,"LAA-950pixel^3":0.0,"LungVol pixel^3":0.0,"LungVol voxs":0.0,"PD15":0.0,"StudyID":NaN,"sex":NaN,"vox volume pixel^3":1.0} |
| Anonymous Sid 10201 | Thorax nativ 3.0 B30f | 20081120 | {"FOV voxels ":4.6923776E7,"LAA-1024%":0.0,"LAA-1024pixel^3":0.0,"LAA-900%":0.0,"LAA-900pixel^3":0.0,"LAA-950%":0.0,"LAA-950pixel^3":0.0,"LungVol pixel^3":0.0,"LungVol voxs":0.0,"PD15":0.0,"StudyID":NaN,"sex":NaN,"vox volume pixel^3":1.0} |
| Anonymous Sid 10113 | LE 3.0 B30f | 20080515 | {"FOV voxels ":5.7933824E7,"LAA-1024%":0.0,"LAA-1024pixel^3":0.0,"LAA-900%":0.0,"LAA-900pixel^3":0.0,"LAA-950%":0.0,"LAA-950pixel^3":0.0,"LungVol pixel^3":0.0,"LungVol voxs":0.0,"PD15":0.0,"StudyID":NaN,"sex":NaN,"vox volume pixel^3":1.0} |
+----------------------+------------------------+------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
select patientname,seriesDescription,studydate,runtable(fetch_dicom_imagej(studyInstanceUID,seriesInstanceUID),"PD LAA Quantify SegmentedLung","") from stolz3pacs where success=true;
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.54504.30000008112007153707800007751 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081120.125332.60 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs4397479100887086077.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.49001.30000008123113283818700000454 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081231.103551.64 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs6777457007696729161.properties --directory emptyDir
Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 12.0 failed 1 times, most recent failure: Lost task 1.0 in stage 12.0 (TID 226, localhost): java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.spark.unsafe.types.UTF8String
at org.apache.spark.sql.catalyst.util.GenericArrayData.getUTF8String(GenericArrayData.scala:55)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown Source)
at org.apache.spark.sql.execution.Project$$anonfun$1$$anonfun$apply$1.apply(basicOperators.scala:51)
at org.apache.spark.sql.execution.Project$$anonfun$1$$anonfun$apply$1.apply(basicOperators.scala:49)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
at scala.collection.AbstractIterator.to(Iterator.scala:1157)
at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:927)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:927)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Driver stacktrace: (state=,code=0)
Stats also causes - Memory - exception:
select patientname,seriesDescription,studydate,stats(fetch_dicom_imagej(studyInstanceUID,seriesInstanceUID)) from stolz3pacs where success=true;
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.49001.30000008123113283818700000454 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081231.103551.64 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs9146868133922682801.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.54504.30000008112007153707800007751 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081120.125332.60 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs8448026936204067464.properties --directory emptyDir
Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 13.0 failed 1 times, most recent failure: Lost task 1.0 in stage 13.0 (TID 228, localhost): java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3236)
at java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:191)
at ch.fourquant.images.types.SerDeserHelper$$anonfun$bytify$1.apply(SerDeserHelper.scala:20)
at ch.fourquant.images.types.SerDeserHelper$$anonfun$bytify$1.apply(SerDeserHelper.scala:15)
at scala.util.Try$.apply(Try.scala:161)
at ch.fourquant.images.types.SerDeserHelper$.bytify(SerDeserHelper.scala:15)
at ch.fourquant.images.types.PipUDT.serialize(images.scala:44)
at ch.fourquant.images.types.PipUDT.serialize(images.scala:16)
at org.apache.spark.sql.catalyst.CatalystTypeConverters$UDTConverter.toCatalystImpl(CatalystTypeConverters.scala:142)
at org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTypeConverters.scala:102)
at org.apache.spark.sql.catalyst.CatalystTypeConverters$$anonfun$createToCatalystConverter$2.apply(CatalystTypeConverters.scala:401)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown Source)
at org.apache.spark.sql.execution.Project$$anonfun$1$$anonfun$apply$1.apply(basicOperators.scala:51)
at org.apache.spark.sql.execution.Project$$anonfun$1$$anonfun$apply$1.apply(basicOperators.scala:49)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
at scala.collection.AbstractIterator.to(Iterator.scala:1157)
at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:927)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:927)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
The memory exception means you need to start java with more memory using the -Xmx
argument. For example java -Xmx8g
will allocated 8gb of memory for java. I am not sure what the default is, but you should make it as high as possible for these tests.
You can also reduce memory usage by adding LIMIT 1
to the sql expression to only get one result for initial testing
Where you able to run some LungQuantification using the anonymized data I posted on the shared drive?
After increasing java memory allocation, the STAT command produces the following error:
select patientname,seriesDescription,studydate,stats(fetch_dicom_imagej(studyInstanceUID,seriesInstanceUID)) from stolz3pacs where success=true;
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.54504.30000008112007153707800007751 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081120.125332.60 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs4259478624266578240.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.49001.30000008123113283818700000454 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081231.103551.64 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs3747596182260056837.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.50021.30000008051505383531200008814 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20080515.113315.4 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs9156190453687388072.properties --directory emptyDir
Error: scala.MatchError: ch.fourquant.images.types.ImageStatisticsUDT@920bd5 (of class ch.fourquant.images.types.ImageStatisticsUDT) (state=,code=0)
Even with increased memory. LungQuantification results are null/zero.
So I have now run the tests on my environment with your data and updated the stats
function to work inside the command line tool. I have also added toarray
to convert the images to an array so you can export them (if needed) and a function to get just a single image fetch_dicom_imagej_single
. Finally there is a new repository which lets you run the tests automatically (https://github.com/4Quant/RIQAE-Java) using JUnit and the Java API for our tools. I have put the new jar and the java tools in the riqae folder and you can run the tests by just typing mvn test
in the root directory
Great! I will test this but likely not until Friday. TJ
As a check for your outputs here are the results from the `PACSCOPDTest
suite integrated into the project
The updated stats returns the basic information about the image (so you can see if it is performing the offset correctly)
Map(mean -> -1988.5996551513672, min -> -4048.0, pts -> 262144.0, max -> 583.0, stdDev -> 1155.2388050224663)
And the results from the COPD plugin aren't null anymore since the calibration is paired to the entire stack and not just the individual slices
[1.3.6.1.4.1.14519.5.2.1.4320.5030.966354075876482042295761929295,
Map(LAA-950mm^3 -> 0.0, LAA-900mm^3 -> 0.0, LungVol mm^3 -> 0.0, PD15 -> -947.0, LAA-900% -> 46.7, StudyID -> NaN, LungVol voxs -> 2171109.0, sex -> NaN, LAA-1024% -> 1.1, LAA-1024mm^3 -> 0.0, LAA-950% -> 13.8, vox volume mm^3 -> 0.0, LungVol (Liters) -> 0.0, FOV voxels -> 1.7825792E7)]
The rest of the results are here
CREATE TEMPORARY TABLE stolz3pacs USING fourquant.pacs OPTIONS (bind "YETI", username "OSIRIX", server "localhost", port "11112", path "StolzAnonymous3.csv");
select patientname,patientID AS StudyID,runmap(run2(fetch_dicom_imagej(studyInstanceUID,seriesInstanceUID),"Segment Lung Tissue",""),"PD LAA Quantify SegmentedLung","") AS Results from Stolz3pacs where success=true;
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.54504.30000008112007153707800007751 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081120.125332.60 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs5403419203677837044.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.49001.30000008123113283818700000454 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20081231.103551.64 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs8807823199842924643.properties --directory emptyDir
PACSToolCall:-L SERIES -c osirix@localhost:11112 -b yeti -m SeriesInstanceUID=1.3.12.2.1107.5.1.4.50021.30000008051505383531200008814 -m StudyInstanceUID=1.2.276.0.18.14.200.2.0.0.20080515.113315.4 --store-tcs /var/folders/cj/p00m_p1137l5wpxrjr5wr62c0000gp/T/store_tcs4370383041495005520.properties --directory emptyDir
+----------------------+-----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| patientname | StudyID | Results |
+----------------------+-----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| Anonymous Sid 10007 | SID10007 | {"FOV voxels ":[8.2313216E7],"LAA-1024%":[0.1],"LAA-1024mm^3":[4411.125],"LAA-900%":[4.1],"LAA-900mm^3":[233690.0625],"LAA-950%":[1.7],"LAA-950mm^3":[98480.8125],"LungVol (Liters)":[5.682],"LungVol mm^3":[5682457.0],"LungVol voxs":[3367382.0],"PD15":[-858.0],"StudyID":[NaN],"sex":[NaN],"vox volume mm^3":[1.6875]} |
| Anonymous Sid 10201 | SID10201 | {"FOV voxels ":[4.6923776E7],"LAA-1024%":[0.1],"LAA-1024mm^3":[4544.734954833984],"LAA-900%":[45.3],"LAA-900mm^3":[3337090.7306671143],"LAA-950%":[8.7],"LAA-950mm^3":[643638.4320259094],"LungVol (Liters)":[7.37],"LungVol mm^3":[7369556.0],"LungVol voxs":[4579283.0],"PD15":[-937.0],"StudyID":[NaN],"sex":[NaN],"vox volume mm^3":[1.6093254089355469]} |
| Anonymous Sid 10113 | SID10113 | {"FOV voxels ":[5.7933824E7],"LAA-1024%":[0.0],"LAA-1024mm^3":[779.0036201477051],"LAA-900%":[9.4],"LAA-900mm^3":[694324.988079071],"LAA-950%":[1.1],"LAA-950mm^3":[83137.51888275146],"LungVol (Liters)":[7.359],"LungVol mm^3":[7359262.0],"LungVol voxs":[3920513.0],"PD15":[-886.0],"StudyID":[NaN],"sex":[NaN],"vox volume mm^3":[1.8771171569824219]} |
+----------------------+-----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
Ground Truth:
PatientName (Anonymized) StudyID LungVol PD15 LAA-950mm^3
Anonymous Sid 10007 SID10007 2.841 -858 49240.431
Anonymous Sid 10113 SID10113 3.68 -886 41568.794
Anonymous Sid 10201 SID10201 4.471 -937 390522.385
Using the lung quantization plugin (as currently installed on my system) directly in Fiji produces legitimate results. However, using the same routines on data retrieved from the PACS using RIQAE produces null or zero or infinite results. Use of another Fiji plugin, "Measure", produces expected results.
Example of Measure command producing legit results:
Example of LungQuant command producing erroneous results: