Add module that computes DRR

[cpinter]

Plastimatch contains an algorithm for DRR reconstruction

Migrated from https://app.assembla.com/spaces/slicerrt/tickets/194-add-module-that-computes-drr/details

[cpinter]

2012-12-07 12:26 Kevin Wang This is a much needed feature for SlicerRT and glad to see it is coming.

2012-12-07 12:49 Greg Sharp It will require some discussion. Should we plan for Utah meeting?

2012-12-07 12:56 Csaba Pinter Yes, we can schedule a discussion at the project week, however I'm pretty sure we can't start working on it until March or so.

2013-09-06 14:54 Csaba Pinter Michi from Vienna AKH has created a DRR module. I tried it and it looks promising. Hopefully it will be published soon.

2015-01-27 15:10 Csaba Pinter @wangk Kevin, is this feature covered in the EBP module?

2015-01-27 15:51 Kevin Wang @pinter, yes, we have a crude version of DRR in EBP module. it is far from ideal but at least we have something working.

2015-01-27 15:55 Csaba Pinter Thanks! So what should happen with this ticket? Whose implementation do we use in the current crude version you mention?

2015-01-28 09:24 Kevin Wang It is my version. I think I will refactor the code and make DRR generation a separate class so the logic can basically switch between different implementations.

2015-01-28 09:24 Kevin Wang is there any news from Michi from Vienna AKH?

2015-01-28 11:17 Csaba Pinter No I haven't heard from them since then.

Separating the DRR computation to its own class is a really good idea, I fully support it!

Thanks!

[MichaelColonel]

Greetings,

What is the status of DRR image computation? I have seen some code in External Beam Plan module, but it commented.

For example i have two points (vtkMRMLMarkupsFiducialNode) that define "virtual" beam direction and normal vector of the x-ray flat-screen detector, normal vector goes through the center of the flat-screen. Position of x-ray source is known, as well as all parameters of the detector itself. I want to calculate DRR image and add it to the plan as a RT Image.

Where should i begin, and what should i do to begin implement DRR calculation?

[cpinter]

This function was implemented long ago and may have been functional in some ways, but was never really used. The lab that implemented it stopped working on SlicerRT, and it was not a priority for anybody else.

It would be an option to revive the commented out code, or use some newer approach. For example it would be possible to make use of this function: https://discourse.slicer.org/t/generate-2d-x-ray-image-from-volume-just-like-drr/976/2

However, the DRR function has never been available from the UI, so I think it would require some planning (hopefully with @lassoan and @gregsharp) before we can decide how it could be most usable.

[lassoan]

You can see an "RT image" equivalent DRR if you render a beam's eye view and enable volume rendering in with the "CT-X-ray" preset. Would this be enough?

[MichaelColonel]

It would be an option to revive the commented out code, or use some newer approach. For example it would be possible to make use of this function: https://discourse.slicer.org/t/generate-2d-x-ray-image-from-volume-just-like-drr/976/2

You can see an "RT image" equivalent DRR if you render a beam's eye view and enable volume rendering in with the "CT-X-ray" preset. Would this be enough?

I would like to start with something simple, where i could use suggested approaches, just to get used to the topic. So i can use the available beam transformation to get beam's eye view, and then generate simple image.

But in the end it must be a part of a some sort of a patient position verification system. So it should be an appropriate UI to generate images like the plastimatch example.

[MichaelColonel]

Greetings,

I've managed to generate a DRR image

CT

DRR

A couple of questions:

1. What is a best way to show some elements (models) of DRR parameters: contour of detector, contour of image, center and normal vector of the detector, marker of the first column and row of the DRR image? I think of separate node which observed RTBeamNode, store some detector parameters and draw all the model elements i needed.

2. Lets say i have a DRR image height, width and raw data buffer, what node should i use to store DRR image in Slicer (ScalarVolumeNode or something else)?

3. How to add a DRR image into SubjectHierarchy correctly (may be there is a ready-made functionality)?

[cpinter]

Hi @MichaelColonel, good to see you again!

1. This is a good question, and I'm not sure about the best approach given that I don't know a lot about how DRRs are usually handled. Now that you are reasonably well versed in the surrounding API, your initial design will probably be close to the optimal one. Please describe in detail what you think would be best, and together we will refine it to be as future proof as possible. @gregsharp do you have any suggestions about this? You know much more about the details of DRRs that we on the computer science side do.

2. (and 3.) There is a Planar Image node that stores RT Image objects loaded from DICOM, which is basically DRR. Please look at this code that creates the planar image node https://github.com/SlicerRt/SlicerRT/blob/master/DicomRtImportExport/Logic/vtkSlicerDicomRtImportExportModuleLogic.cxx#L1321-L1420 and this, setting up geometry https://github.com/SlicerRt/SlicerRT/blob/master/DicomRtImportExport/Logic/vtkSlicerDicomRtImportExportModuleLogic.cxx#L1564-L1830 Subject hierarchy is set up mostly in the first function but also managed in the second one.

[gregsharp]

I am not aware of any standard format for those elements. There is no DICOM equivalent for things like structure outlines or crosshair. Some TPS use a "burn in" approach, which allows their appearance on the RT Image, but that does not allow colored structure outlines.

If you want to go with model elements instead of burning them into the image, I expect they should be grouped with the RT Image, within a new MRML node type. Or alternately they are contained within the RT Image node.

[lassoan]

If you want to show the DRR in a slice view then I would recommend to use markups to display points, lines, and curves over it.

[MichaelColonel]

Thank you for the comments,

All the models elements are just for the visualization of detector and image boundaries in 3D, and "burn in" approach doesn't needed (i hope).

I would like to use plastimatch drr parameters and only single image mode for beginning. Some parameters will be taken from beam node (SAD, direction) others will be set in GUI (SID, detector dimension, resolution, etc).

Showing DRR in a slice view is a good option, and will think about using markups.

[MichaelColonel]

While converting precalculated DRR from vtkMRMLScalarVolumeNode into vtkMRMLPlanarImageNode what values allowed for DICOM UID values in subject hierarchy node? Will the empty string "" will be enough, since a new DRR image is not part of the DICOM series?

e. g. values here

[cpinter]

If it was not loaded from DICOM then its UID is empty until we export it to DICOM. I think instead of setting it to empty string you should actually not set it at all.

[MichaelColonel]

@cpinter another question.

How to use vtkSlicerPlanarImageModuleLogic in a loadable module logic?

When i initialize with macros

vtkCxxSetObjectMacro(vtkSlicerPlmDrrLogic, PlanarImageLogic, vtkSlicerPlanarImageModuleLogic);

the pointer to logic seems to be valid, but the the program crashes.

1. vtkSlicerPlanarImageModuleLogic pointer in DRR logic header;

2. Initialization macros in a source file;

3. Place where it crashes. Two methods (LoadRtImage and SetupRtImageGeometry) were taken and modified from vtkSlicerDicomRtImportExportModuleLogic.

[cpinter]

Make sure your module includes the Planar Image logic target https://github.com/SlicerRt/SlicerRT/blob/master/DicomRtImportExport/CMakeLists.txt#L54 and depends on the module https://github.com/SlicerRt/SlicerRT/blob/master/DicomRtImportExport/qSlicerDicomRtImportExportModule.cxx#L111

Also that the module class sets the pointer to the DRR logic https://github.com/SlicerRt/SlicerRT/blob/master/DicomRtImportExport/qSlicerDicomRtImportExportModule.cxx#L133-L143

[MichaelColonel]

Greetings,

The loadable module for DRR calculation using plastimatch via QProcess is almost ready for initial review. Initially only single image mode available.

Does the plastimatch have a command line version for Windows and MacOS, so anyone could test it?

[lassoan]

It will be great to have this feature!

Note that you don't need QProcess to launch external command-line applications. All you need is to write an xml file that describes command-line parameter types and names (https://www.slicer.org/wiki/Documentation/Nightly/Developers/SlicerExecutionModel) and Slicer can export all input data from the scene in the requested format, run the application (optionally in the background), waits for the execution to complete, and reads back all output data into the scene. Slicer automatically generates a GUI from the XML file that allows setting all inputs and outputs and start/monitor/cancel execution, and you can also use the module from other C++ or Python scripted loadable modules. You can of course implement all these manually - writing out nodes to files, launching the application, monitor its status, read back results, create GUI, etc., but it is much less development and maintenance work if you use CLI module infrastructure.

See this example how to make a third-party command-line tool a Slicer CLI module: https://github.com/SlicerHeart/SlicerHeart/tree/master/ConformalTextureMapping

PlmLandwarp, PlmSynth, PlmThreshbox in SlicerRT follow a similar approach but they provide their own main function, too, not just an XML for an existing main function.

[MichaelColonel]

I know about CLI module, but i tried to make it WYSIWYG: beam node for a normal and view-up vectors calculation, volume node for the data itself.

As i understood correctly the plastimatch build for SlicerRT doesn't have a plmreconstruct module by default and drr_compute function is unavailable. Parsing of the command line arguments for Drr_options is also a little bit tricky.

Next time for such purpose i will use CLI module.

[lassoan]

If you want faster turnaround time then use plastimatch library directly in a loadable or scripted module. If you want to decouple DRR generation (so that it can run in the background) then probably CLI is the simplest. You can run the CLI from your loadable or scripted module if you want more intuitive GUI.

You can edit External_Plastimatch.cmake to change what parts of plastimatch are included in SlicerRT.

It is still not too late to change anything. Send the pull request and we'll see what seems to be the best way to proceed.

[MichaelColonel]

You can edit External_Plastimatch.cmake to change what parts of plastimatch are included in SlicerRT.

I've done that, but the plastimatch for DRR computation uses MetaImageHeader (*.mha) file for the input, and raw file for the output. So even using plastimatch library directly i have to save scalar volume to the temporary file and after computation again load the result from the file, and these I/O operations are much slower than the possible QProcess overhead.

There is no other option.

The only solution is add another, modified drr_compute, that will allow to use ITK images as input and output data.

[cpinter]

The only solution is add another, modified drr_compute, that will allow to use ITK images as input and output data

Instead of duplicating stuff we can modify the existing one. It shouldn't be too hard, given that Plastimatch uses ITK internally. @gregsharp Do you have any thoughts on this?

[lassoan]

So even using plastimatch library directly i have to save scalar volume to the temporary file and after computation again load the result from the file, and these I/O operations are much slower than the possible QProcess overhead.

File reading/writing takes about 1s for a large volume (if compression is disabled), which is typically negligible overhead if there is any significant processing. This is reduced to 0s if you use CLI modules and disable Application Settings / Modules / Prefer executable CLIs, as in this case the "file IO" is overridden in the background and converted to an in-memory passing of a pointer. However, we usually keep "executable CLIs" option enabled to isolate the application from CLI module errors (CLI modules cannot crash the application) and to make it possible to abort CLI modules at any time.

So, if performance is critical then it is probably better to use plastimatch library directly and pass the image in memory, taking the image from memory instead of loading from file. If plastimatch library does not have an API to pass in an image then it should not be hard to change that.

[MichaelColonel]

Performance isn't critical, and i can rework logic to use drr_compute instead of QProcess. The main issue in CLI module is that one can't see a "feedback" of what he is doing: no imager border or image window changes, no vectors, isocenter-imager distance, etc.

As a solutions the drr computation can be implemented as a CLI module with MRML and plastimatch options, where I/O operations will take place, and then use that module in a loadable module for example for proper visualization. IMHO.

[lassoan]

Yes, nowadays we implement a CLI modules to take care of data marshalling, process execution, etc. and if the auto-generated CLI GUI is not satisfactory (which is often the case) then we just create a very lightweight Python scripted module that has a nice GUI.

[MichaelColonel]

What is a better way of using vtkMRMLPlanarImagerNode for DRR data storage?

1. Add members and methods into planar node;
2. Use vtkMRMLPlanarImagerNode as a base class for a vtkMRMLRtImageNode for example.

[cpinter]

I suggest doing 2: subclassing the planar image node. It is cleaner to start with, and if we decide that the planar image node is not needed separately we can remove it in the future.

[MichaelColonel]

CLI and GUI modules are ready for review in #161.

[MichaelColonel]

Is there any reason adding RTK forward projection filter as an alternative for DRR calculation algorithm? Benefits: Straight forward ITK filters, not need for addional I/O operations, almost instant calculation. Drawbacks: Additional external library for compilation.

Here is an example of RTK result: