Feature request: Support for DJI Mini3 DNGs lens correction

[WiLars]

Is your feature request related to a problem? Please describe. The DJI Mini3 drone generates DNGs that have embedded lens correction profiles. These profiles are not applied on import or in the lens correction module. Correction using Lensfun is not possible, because the vignette is not only related to luminance but also has a significant color shift (/lensfun/lensfun/issues/1963). This color shift basically makes the raws unusable in Darktable.

Describe the solution you'd like Apply the lens profiles embedded in the DNG files in the context of the lens correction module

Alternatives Preprocessing the DNG files, so that Darktable doesn't have to apply correction profiles. I'm actually not sure how I could do that.

Additional context I think the foundation for this was implemented in darktable-org/darktable/pull/12714 . This functionality works great for my camera, but not for DJI Mini3. @paolodepetrillo, do you have some additional insights why this does not work? As you shortly indicated in this comment you already looked into DJI drones? It looks like you have a deep understanding of OpcodeList in the DNG file format.

[Nilvus]

@sgotti: what's your view on that?

@WiLars: given your FR, darktable version related actually seems obvious but think about precising it anyway in future FR/issues. There's a template that request this and it's there on purpose. Thanks!

[WiLars]

@Nilvus thanks for the hint. I think in the template for feature requests doesn't request that info, so i wasn't aware. Anyway, here is some additional info, as found in the bugreport template:

Platform

• darktable version : 4.2.0
• OS : Linux Kernel 5.15.85-1-MANJARO
• Linux - Distro :Manjaro
• Memory : 32 Gb
• Graphics card : Nvidia RTX 3060Ti
• Graphics driver : 525.60.11
• OpenCL installed : no
• OpenCL activated : no
• Xorg : X.Org X Server 1.21.1.6
• Desktop : KDE installed from repo, so i think built parameters do not really apply?

Additional context

• Can you reproduce with a RAW or Jpeg or both? only raw, in jpeg lens corrections are already applied by drone
• Are the steps above reproducible with a fresh edit (i.e. after discarding history)? yes
• If the issue is with the output image, attach an XMP file if (you'll have to change the extension to .txt)
• Is the issue still present using an empty/new config-dir (e.g. start darktable with --configdir "/tmp")? yes
• Do you use lua scripts? no

I think i can't just upload a raw file as example here, but if anyone is interested I can provide some.

[Nilvus]

@Nilvus thanks for the hint. I think in the template for feature requests doesn't request that info, so i wasn't aware. Anyway, here is some additional info, as found in the bugreport template:

You're right because for feature request, most of infos of bugreport template are not needed (included for most FR requests) darktable version. Your request is something between camera support and feature request. And for camera support, we often have some request with existing issues or fixed in a previous release.

[kmilos]

Please upload samples to raw.pixls.us, and also share here via some cloud service.

[WiLars]

here you can find one actual photo and a white one I took for my experiments with lensfun. The white one is a photo of a white monitor taken through a diffusor, so it is supposed to be uniformly white. In that one you can clearly see the color shift

I also uploaded the landscape top-down to raw.pixls.us

[kmilos]

So, there is

OpcodeList3                     : FixVignetteRadial, GainMap

and "should be applied to the raw image, just after it has been demosaiced" (which is unsupported by dt currently obviously).

Edit: Neither of these is specified as optional; FixVignetteRadial applies to all color planes per definition, while GainMap is a per-plane 65x49 map...

[spaceChRiS]

Related: https://github.com/darktable-org/darktable/pull/12880 As GainMap is already implemented for OpcodeList 2 (pre-demosaicing), but is here used in OpcodeList 3 (post-demosaicing), the OpcodeList would have to be distinguished in the internal metadata structure if I understand it correctly.

[kmilos]

As GainMap is already implemented for OpcodeList 2, but is here used in OpcodeList 3, the OpcodeList would have to be distinguished in the metadata if I understand it correctly.

Not only metadata (which is the easy part) - it has to be applied in the correct place in the pipeline - OpcodeList2 is applied to single-channel raw CFA data (currently implemented in the rawprepare module right after the file is opened); OpcodeList3 must be applied just after demosaic and applied to 3 color channels, and this is currently not implemented.

Not 100% sure we can even apply the FixVignetteRadial OpcodeList3 in the lens correction module (which could've been relatively easy to extend), as it might be in the incorrect place in the pipeline wrt to the DNG spec...

And then there is the crop issue @jenshannoschwalm mentioned...

[jenshannoschwalm]

I have pending work related to dng lens correction and indeed that should be as early as possible after demosaicing (or sort of Post-Processing there)

[kmilos]

It gets somewhat more complicated - this sample seems to have a (weird?) 2x digital zoom scheme: crops out middle 2000x1500, then scales pixels by 2x to get back to "expected" 4000x3000 output...

What is not clear is when the OpcodeList3 is applied - it could be to the whole active area before this crop+rescale because the FixVignetteRadial is unchanged? (The GainMap is again on a 4000x3000 size, but that doesn't tell me it's before or after.)

[paolodepetrillo]

@kmilos From that sample gallery of 4 images, some have the 2x digital zoom but 1xsingle.DNG does not. The GainMaps for all of them regardless of the 1x / 2x zoom are similar, which indicates that the OpcodeList3 should be applied to the entire image before the crop. If it was intended to be applied after the crop, the GainMap for the 2x image would look like a cropped version of the GainMap from the 1x image.

[kmilos]

Indeed, as the identical FixVignetteRadial suggests as well...

[paolodepetrillo]

@WiLars As a short term workaround until this feature is implemented in darktable, you can use my dngpreprocess utility to apply the corrections outside of darktable, resulting in a new dng file that already has the corrections applied and can be edited in darktable.

Instructions to build and use are here: https://discuss.pixls.us/t/pixel-7-pro-raw-processing/33942/32

The "-3" option must be specified to apply the OpcodeList3 processing for the DJI files: dngpreprocess -3 DJI_9999.dng

[WiLars]

@paolodepetrillo thank you for the hint, just tried it, and with your dngpreprocess the raws look much better!

I addded some exported files (jpeg) to the folder. There is still some shade on top and bottom and some slight color shift, but I suppose this is related to the quality of the embedded profiles...

[github-actions[bot]]

This issue did not get any activity in the past 60 days and will be closed in 365 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

[andersnygaard]

DJI just released it's Air 3 drone using the same 24mm camera and sensor as the Mini 3. I get close to if not the same result from importing the pictures from this camera. However, the drone has a secondary 70mm camera built on the same CMOS architecture and pictures from this camera does work in Darktable, applying lens correction just fine.

Compare files from the two uploaded to my Onedrive at https://1drv.ms/f/s!AtaerbceABlrie5qp45Iwrg9JNABRQ?e=EPWHsF

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

[jchnkl]

@WiLars As a short term workaround until this feature is implemented in darktable, you can use my dngpreprocess utility to apply the corrections outside of darktable, resulting in a new dng file that already has the corrections applied and can be edited in darktable.

Instructions to build and use are here: https://discuss.pixls.us/t/pixel-7-pro-raw-processing/33942/32

The "-3" option must be specified to apply the OpcodeList3 processing for the DJI files: dngpreprocess -3 DJI_9999.dng

Is there any ongoing work to incorporate this in darktable / libraw? With the latest darktable version I have access to (4.4.2) there is a significant difference between the images before and after running them through @paolodepetrillo 's dngpreprocess:

Screenshot


Unfortunately this roughly doubles the file size and requires me to store a second file together with the original which is not an ideal situation.

[jenshannoschwalm]

Is there any ongoing work to incorporate this in darktable / libraw?

Not as far as i know. Bye me a new leica using the opcode3 on it's dng files and i will implement it immediately :-)

[jchnkl]

Is there any ongoing work to incorporate this in darktable / libraw?

Not as far as i know. Bye me a new leica using the opcode3 on it's dng files and i will implement it immediately :-)

A DJI Mini 3 will do ;)

Joking aside: Could you provide me with some starting points where the relevant code is located (libraw or darktable, maybe even on file level?). Then I could check if it's feasible to adapt the code from the dngpreprocess util.

Or would that be a no go anyway because it links to the Adobe SDK?

[paolodepetrillo]

I don't think it would be worthwhile to refer to the Adobe DNG SDK code from dngpreprocess - the GainMap is a simple operation that's fully documented in the Adobe DNG Spec.

The first step is to read the OpcodeList3 GainMaps out of the file and store them in dt_image_t to make them available to processing modules. There is an existing field dng_gain_maps that holds the OpcodeList2 GainMaps and a new field would be added for the OpcodeList3 ones. In dng_opcode.c, dt_dng_opcode_process_opcode_list_3 would also process GainMaps like dt_dng_opcode_process_opcode_list_2 does.

Then we need to figure out where in the processing pipeline to apply the correction. It could be done inside lens correction as it's at the correct place in the pipeline, although that module is already complicated.. or it could be done in a new module.

Another possibility is to convert the post-demosaic OpcodeList3 GainMap to a pre-demosaic OpcodeList2 GainMap having almost the exact same effect, and apply it using the existing implementation in rawprepare. This might not be strictly compliant with the spec, but I think the results would probably be visually indistinguishable and it's worth trying.

[jchnkl]

@paolodepetrillo Thank you very much, that's very helpful! The first part sounds pretty straight forward. I'll look into reading the opcodes and then putting them into the gain maps structure. From there on we can experiment to figure out what's the best place for the correction.

[Mr-Groch]

Hi! I've also suffering from this issue as Mini 3 owner. Are there any plans to put this on roadmap?

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

[Mr-Groch]

I don't see Mini 3 support added in master branch

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

[Mr-Groch]

Still no Mini 3 DNGs support

[jenshannoschwalm]

Still no Mini 3 DNGs support

We know :-) It very much seems there is no dt developer currently interested in this sort of correction, i suppose none are using such devices.

Just two reminders:

1. Such "missing support for my camera" started me to get into darktable code. A lot of code to be reused is there, there are already some hints, just go ahead :-)
2. Some comments above i suggested something ... :-)

[kmilos]

Still no Mini 3 DNGs support

4.7.0+1332~geeb6ad3f8d here opens the above linked files just fine, and I see the embedded vignette correction if I enable the lens correction module:

OpcodeList3 GainMap is the remaining thing missing.