Evaluate functionality provided by Mayo Grunt and consider adding it as official interface

[fedorov]

As communicated by @potis (thanks!):

The most recent version of the dcmqi-grunt docker is available.

The github repo is availble at

https://github.com/Mayo-QIN/dcmqi

And the docker image can be downloaded from docker hub.

docker pull mayoqin/dcmqi

[fedorov]

@potis @blezek can you clarify a bit - how should the arguments that take multiple files be handled? The issue is that in some cases (specifically, for itkimage2segimage), the order of files matters. So even if your UI can take a zip file as input, it will not always work, because it is not clear how files would be sorted after being extracted. Is there anything you have in grunt that can help while passing arguments that contain ordered lists of files?

[fedorov]

[blezek]

@fedorov, interesting use case! We hadn't considered this particular use case. In the past, we've had to write simple "shims" in bash to handle cases like this. A comma-separated list of filenames isn't a typical way of passing multiple files, and grunt doesn't know how to handle them. If the order doesn't matter, you could use the shim to accept a directory, then format the filenames in the directory to pass into the final command. Not perhaps the way you wanted to go, but it would work.

[fedorov]

@blezek do you have documentation about "writing simple "shims""? I do not understand what this means. As I explained, the order does matter.

As an aside, what do you consider a typical way of passing multiple files?

[blezek]

@fedorov, itkimage2segimage has a --inputImageList of type std::vector<std::string> that takes a list of comma-separated image filenames. Currently grunt doesn't know how to handle this input (and since it's not common, we may not handle it ever). However grunt does know how to handle a directory full of images. To address this, create a shim, or thin bash script translating what grunt knows how to do to what itkimage2segimage needs done. For example, the shim would take a command line like this: shim --inputImageList <directory> --inputDICOMDirectory <directory> --inputDICOMList <directory> --outputDICOM <directory>. The corresponding grunt description would be:

services:
  - endPoint: itkimage2segimage
    commandLine: ["shim", "--inputImageList", "^images", "--inputDICOMDirectory", "^dicom", "--inputDICOMList", "^dicom_list", "--outputDICOM", "~output"]

The shim script would join the files in images, dicom and dicom_list into a comma-separated string, and pass that to the itkimage2segimage command. A parameter starting with ^ tells grunt to expect an uploaded zip file containing files, and a parameter starting with ~ tells grunt to create a directory to contain output files. The ~ arguments are available for download as a zip file after the job is finished.

Does this help?

[blezek]

@fedorov, @potis, Added some more documentation here: https://github.com/Mayo-QIN/grunt/tree/parameter_documentation#shims

[potis]

@fedorov, if there are sample files i can update the dcmqi grunt and test it to make sure it will work as expected.

[fedorov]

@potis @blezek thanks for the further clarifications. I hope we can make it work!

The issue that is still unclear to me is how would we communicate the order of files from the user to the processing tool, or the shim? There are no references to the specific files by name from the JSON file. Each input segmentation is matched by a metadata entry in the JSON file as determined by the order of that file in the list.

Does it mean we need to change the way dcmqi handles this? Did you consider adding a functionality where files could be added one by one to an argument that requires a list of files?

[fedorov]

Here is an end-to-end example that can be used to illustrate/test the situation with multiple input segmentations passed to the converter: https://qiicr.gitbooks.io/dcmqi-guide/content/use_cases/freesurfer.html

[blezek]

@fedorov, I finally see the problem(forgive me for being dense). Would there be some way to add the filename in the JSON? Depending on the order of files on the command line to be the same as the order in the JSON file might end up being a little fragile.

@potis and I are in the office together tomorrow, would you like to talk about this issue?

[fedorov]

@blezek of course it is possible to add filenames to the schema. The only concern is backwards compatibility, but we probably don't have that many users at that point, and also I agree it is somewhat fragile the way things are done now. From my personal experience, it definitely can be error prone and painful to edit the metadata in JSON file without explicit references to the files individual sections of JSON correspond to.

I will follow up by email about scheduling to talk tomorrow.

[fedorov]

@nolden FYI (since you are the user!) - let us know if you have any concerns related to modifying the schema for the SEG converter JSON files.

[nolden]

lgtm

[fedorov]

Pending response from @blezek on #253