[Question] DICOM Pixel data manipulation

[WoonchanCho]

Hi team, This is not an issue at all but a question.

The dcmjs library works pretty well to manipulate the DICOM metadata. I wondered if you have a plan to add functions to easily manipulate pixel data (e.g, burning-in some portion of pixel data). I take some time to review the DICOM standard and found out that it's quite not easy to implement from the scratch because DICOM supports many types (bmp, jpeg, multiframe, ...) of pixel image type.

May I know if you plan to add a feature to manipulate DICOM images of various types? If this is not in your interest, may I know if you know if there are any other javascript libraries that does something like this?

[pieper]

Hi @WoonchanCho -

It makes sense to do some basic pixel manipulation in this library, particularly related to codecs, but not image processing per se. The dicomParser project is more focused on the reader part of the process and covers more of that side of things while dcmjs is more focused on the metadata, as you say. We should be able to share the webassembly codec, but I haven't looked into that.

For more general processing, itkjs could be good, and so is imagemagic. Both are available in webassembly.

I've also used dcmjs in the context of webgl experiments, going from dicom to dicom via the GPU (https://github.com/pieper/step) which was kind of my motivation for dcmjs in the first place, but that kind of processing will always be a layer higher on top of dcmjs.

Maybe we could make a list of the pixel manipulations are are proper to dicom, like transfer syntax support, and see how best to implement them.

[WoonchanCho]

Thank you so much for your answer @pieper. I agree with your opinion that it would be good to add some basic pixel manipulation logic to this library not image processing. I will be happy to contribute to adding this feature to this library. I will review the libraries you mentioned first

Thank you so much,

[WoonchanCho]

@pieper It looks dicomParser (with cornerstoneWADOImageLoader) supports the following transfer syntaxes: https://github.com/cornerstonejs/cornerstoneWADOImageLoader/blob/00cf0419c98c6bddef121d4409c8a2a9d780f336/src/imageLoader/decodeImageFrame.js#L62

1.2.840.10008.1.2 Implicit VR Little Endian 1.2.840.10008.1.2.1 Explicit VR Little Endian 1.2.840.10008.1.2.2 Explicit VR Big Endian (retired) 1.2.840.10008.1.2.1.99 Deflate transfer syntax (deflated by dicomParser) 1.2.840.10008.1.2.5 RLE Lossless 1.2.840.10008.1.2.4.50 JPEG Baseline lossy process 1 (8 bit) 1.2.840.10008.1.2.4.51 JPEG Baseline lossy process 2 & 4 (12 bit) 1.2.840.10008.1.2.4.57 JPEG Lossless, Nonhierarchical (Processes 14) 1.2.840.10008.1.2.4.70 JPEG Lossless, Nonhierarchical (Processes 14 [Selection 1]) 1.2.840.10008.1.2.4.80 JPEG-LS Lossless Image Compression 1.2.840.10008.1.2.4.81 JPEG-LS Lossy (Near-Lossless) Image Compression 1.2.840.10008.1.2.4.90 JPEG 2000 Lossless 1.2.840.10008.1.2.4.91 JPEG 2000 Lossy

Additionally the same library doesn't officially the following syntaxes 1.2.840.10008.1.2.4.92 JPEG 2000 Part 2 Multi-component Image Compression (Lossless Only) 1.2.840.10008.1.2.4.93 JPEG 2000 Part 2 Multi-component Image Compression

Following are not supported either but they are not image format. 1.2.840.10008.1.2.4.100 MPEG2 Main Profile @ Main Level 1.2.840.10008.1.2.4.101 MPEG2 Main Profile @ High Level 1.2.840.10008.1.2.4.102 MPEG-4 AVC/H.264 High Profile / Level 4.1 1.2.840.10008.1.2.4.103 MPEG-4 AVC/H.264 BD-compatible High Profile / Level 4.1 1.2.840.10008.1.2.4.104 MPEG-4 AVC/H.264 High Profile / Level 4.2 For 2D Video 1.2.840.10008.1.2.4.105 MPEG-4 AVC/H.264 High Profile / Level 4.2 For 3D Video 1.2.840.10008.1.2.4.106 MPEG-4 AVC/H.264 Stereo High Profile / Level 4.2 1.2.840.10008.1.2.4.107 HEVC/H.265 Main Profile / Level 5.1 1.2.840.10008.1.2.4.108 HEVC/H.265 Main 10 Profile / Level 5.1

[pieper]

Those look like the most common transfer syntax use cases to me.

Of course we will also want test cases for those. I've started brainstorming about how we might best make them available as they scale, ideally migrating away from the current release asset method used in dcmjs-org/data.

I'm hoping we'll be able to set up a DICOMweb service somehow to host these examples for testing but also for demos and tutorials. There are some notes here: https://github.com/pieper/dicomzoo

[WoonchanCho]

@pieper I've played burning-in an area in few samples dicom images of few transfer syntaxes and saving the changes. it worked. A goal of this library supports both web and nodejs environments. So, do you think that the pixel data manipulation is run on web worker in the web and on the different threads (worker thread) in the nodejs?

The dicomzoo project would be very useful. It would be great if there is some category defined e.g by modality (or SOP class UID), transfer syntax, normal or error case. etc.

[pieper]

@pieper I've played burning-in an area in few samples dicom images of few transfer syntaxes and saving the changes. it worked.

Great!

A goal of this library supports both web and nodejs environments. So, do you think that the pixel data manipulation is run on web worker in the web and on the different threads (worker thread) in the nodejs?

I would think yes, although I haven't tried it myself. Although one could use native code on node, I think it would be cleanest to aim for one codebase with web assembly and web workers that works in both places. I'd prefer that unless there were performance or other drawbacks on node.

The dicomzoo project would be very useful. It would be great if there is some category defined e.g by modality (or SOP class UID), transfer syntax, normal or error case. etc.

Agreed. Debugging dicom code is always painful because people expect every variant to work even though there's no test data for it.

[WoonchanCho]

@pieper Sounds good. As far as I know, web workers don't not work on Node.Js env, right? If so, if we don't use native worker thread on NodeJs, the the pixel manipulation process will need to be run on the main thread on NodeJs. Is this what you expected? I don't know how this will affect NodeJs app because I dont see (close-to-production ready) examples to use this library on Node.

Please advise me if I miss something.

[pieper]

I haven't worked with it so I don't know if there are drawbacks, but I was thinking that something like this web-worker npm package would abstract away the differences between the node and browser use cases. Might make the code easier to develop and maintain.

[WoonchanCho]

@pieper thank you for your comment. I think we are on the same page. I also saw that library too and considered to use it or its alternative. I'll try to do implement something minimally working and let you know soon.