Parser performance

[jcupitt]

On a 720mb Leica DICOM, git master libdicom takes about a second to parse the file::

$ time dcm-getframe -v -o x.jpg 1.3.6.1.4.1.36533.992031277215326518414019311377114211255162.dcm 1
INFO     [Thu Feb 16 10:33:49 2023] - Read file '1.3.6.1.4.1.36533.992031277215326518414019311377114211255162.dcm'
INFO     [Thu Feb 16 10:33:49 2023] - Read metadata
INFO     [Thu Feb 16 10:33:51 2023] - Read BOT
INFO     [Thu Feb 16 10:33:51 2023] - Basic Offset Table is empty.

real    0m1.355s
user    0m0.805s
sys 0m0.549s

Optimisation ideas in no order:

• It's spending about 10% of runtime in strcmp(), according to callgrind. We could swap out the string VRs for an enum.
• dcm_offset() (get current file pointer) flushes the input buffer unnecessarily, we could add a special path for this. Though it doesn't seem to be called in the main parse.
• It's spending 16% of runtime in int_malloc and int_free, suggesting there are many small mallocs. Some of these could probably be moved to the stack.
• 5% of runtime is in eheader_check_vr(), which should not really exist.

So I suppose there might be a 20% speedup possible.

[hackermd]

Thanks for the deep dive and detailed analysis @jcupitt. Unfortunately, large multi-frame images with TILED_SPARSE dimension organization are a pain to work with and we have problems with it at multiple fronts. See for example https://github.com/ImagingDataCommons/highdicom/issues/202.

Having a highly performant C implementation would thus be extremely useful.

cc @CPBridge @dclunie @pieper

[jcupitt]

Yes, I've been using fo-dicom (a nice C# dicom loader), but it also really struggles with TILED_SPARSE slide images. It can take a minute or so to load a large one.

Leica assume me they are planning to support TILED_FULL, so ... fingers crossed. I think their main issue is with row-major vs column-major tile layout. Writing a BOT would help too, of course.

I've started an experimental performance branch to see if I can get libdicom running a bit quicker.

[jcupitt]

OK, implemented with:

https://github.com/ImagingDataCommons/libdicom/pull/43

It's about 20% faster.