Improve DNG Conversion Performance

[Alex-Vasile]

Currently if you want to test out different settings for the same non-DNG input raw files the intermediary DNGs will be deleted at the completion of each save. Further, I don't think that the conversion of multiple files is occurring in parallel.

Here's timing number for merging 30 files using the higher-quality merging approach on my m1 MacBook Pro. I'm using 30 here to get a good estimate of the per-file time, this would itself represent a pretty big merge.

The conversion to DNG is using up 20% of that run time. The relative time will be even worse for the spatial merge, or if the simple temporal averaging is used. Stage	Time [s]	Time [%]
Time to convert images:	6.8	20.1%
Time to load all images:	5.1	14.9%
Time to align+merge all images:	19.8	58.1%
Time to save final image:	2.35	6.9%
Total Time	34	100%

I think that we can get an easy speed improvement to the app by changing the behaviour to:

• Don't delete tmp files until app is closed (and delete anything in there on startup to ensure files don't accumulate if the app closes in a way that prevents it from cleaning up.
• Before running the raw->dng conversion on each file, check if it already exists in the cache folder.
• Keep a cap on the max folder size. Say, start deleting files when it gets over 2GB.
• See if there's a way to mark it for the temp folder and have the OS clean it up for us.

The time to read them from file is a further 14.9% of the run time. As a second step. Caching the loaded files in memory (maybe 0.5-1GB) would reduce the run time by the whole 35%.

The same think for the output, it seems like a shame that the DNG has to be written to a file only to be immediately read back and ran through the Adobe DNG Convertor (if present).

[chris-rank]

I double-checked the code and the two categories "FindNewRawImageDigest" and "Write raw image time" are already included in the "Time to save final image". It is a bit confusing as some printouts are from the DNG SDK. Today, I added a console printout with "Total processing time" to make performance comparisons easier. I fully agree that the DNG conversion at the beginning should be performed only once per app session (with some max "cache" size). This would improve user experience and save processing time.