Beep6581 / RawTherapee

A powerful cross-platform raw photo processing program
https://rawtherapee.com
GNU General Public License v3.0
2.74k stars 313 forks source link

RAWTherapee DJI mini 4 Pro DNG #6995

Closed RAWimageViewer closed 5 months ago

RAWimageViewer commented 5 months ago

ISSUE: DNG photo first loaded into RT 5.10 appears light in center and darker around. This is not the way the JPEG corresponding image appears. Why does the DNG image appears as uneven exposure. This is what happens for all the DNG photos loaded into RT 5.10

DNG RAW photo source: The supplied images are from (screenshots 1 and 2) from a DJI Mini 4 Pro camera set to AUTO and capturing JPEG + RAW photo

1) RT VERSION:

Version: 5.10 Branch: 5.10 Commit: 8adb5439d Commit date: 2024-02-15 Compiler: clang 15.0.0.15000100 Processor: arm64 System: Apple Bit depth: 64 bits Gtkmm: V3.24.8 Lensfun: V0.3.4.0 Build type: Release

2) Apple operating system name and version:

Sonoma 14.4

3) steps to reproduce

Simply move the photo files from the drone SD card to the laptop computer (Apple M2 Pro processor chip) hard drive and place in a folder. Open RT and then direct RT to that folder to see the photos. Clicking on/selecting a DNG photo results in the center being a brighter part of the image and the surrounding area is darker.

Screenshot 1 DNG has had no edits or corrections, just appears when loaded in preparation to make adjustments.

Screenshot 2 JPG is the same corresponding drone photo when the drone captures a pair of photos (one is JPEG and the other is the RAW DNG photo)

When opening the same DNG file using the Apple PREVIEW function one sees the next screenshot type image:

Screenshot 3

4) Uploaded files:

Screenshot 1 Screenshot 2 Screenshot 3

GitHub does not allow file types to upload including DNG and pp3: DNG photo (DJI_20240314104659_0127_D) Pp3 JPEG photo (DJI_20240314104659_0127_D) pp3

Added Note: RT does presently include camera settings for some of the DJI drones, but it might be nice to add others like the mini 4 pro. If it helps you, might I contact their technical support team and request they provide you info for the Hasselblad mini 4 pro drone camera?

(Added note2: A similar issue was illustrated by another person on a Reddit comment: https://www.reddit.com/r/dji/comments/17c8hw3/mini_4_pro_editing_dng_with_darktable_or/ )

Screenshot 1 DNG photo Screenshot  2 JPG photo Screenshot 3 dng preview

Lawrence37 commented 5 months ago

Duplicate of #6868.

What you are seeing is vignetting. It's especially pronounced in many small sensor cameras like drones and smartphones (in my experience). What are the "camera settings for some of the DJI drones" you are talking about? You can use filebin.net to share large files. GitHub accepts smaller files as long as the extension is supported (rename xyz.pp3 to xyz.pp3.txt or put them in a .zip file before uploading).

RAWimageViewer commented 5 months ago

Lawrence First, please accept that I am new user of RT and even though have a scientist profession, I probably need to learn much more about RT. I apologize for any misunderstanding, but know I am writing to help and my words are upbeat and with some enthusiasm! Second, yes, the appearance of the loaded DNG photo is with some vignetting, yet a long attempt to moderate that kind of characteristic makes working with RT rather incomplete … especially compared to the even exposure in a JPEG made by the exact same camera! Third, I referred to a camera setting for DJI cameras and I found that under “Profiled Lens Correction” and I have added two screen shots here in that regard. The pop up menu shows which DJI cameras are currently supported. Maybe this setting is helpful in countering the vignetting effect, or maybe not so helpful, but at least this was what I was referring to! Fourth, various YouTube videos (focusing on RAW DJI Mini 4 Pro drone photos) with Adobe or other Vender software used to edit and process DNG files seem not to have the same vignetting effect that has been observed here (and by the other user on Reddit) when loading the photo into the first step before doing any adjustments. Fifth, the files are uploaded to https://filebin.net/m7r5hsvvrfzlvt6s Summary: I hope this helps you and any possible further development of RAWTherapee. I wish to be helpful if what I am sharing is in fact something that needs attention. I will say the photo I have shared is one of many that all exhibit the same type of appearance once loaded into RT. And I hope that working with RT becomes very helpful and a great experience. Todd

On Mar 16, 2024, at 5:56 PM, Lawrence37 @.***> wrote:

Duplicate of #6868 https://github.com/Beep6581/RawTherapee/issues/6868.

What you are seeing is vignetting. It's especially pronounced in many small sensor cameras like drones and smartphones (in my experience). What are the "camera settings for some of the DJI drones" you are talking about? You can use filebin.net to share large files. GitHub accepts smaller files as long as the extension is supported (rename xyz.pp3 to xyz.pp3.txt or put them in a .zip file before uploading).

— Reply to this email directly, view it on GitHub https://github.com/Beep6581/RawTherapee/issues/6995#issuecomment-2002150617, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHARQ5RQVPBWOR4J4MWYJJDYYS5ZNAVCNFSM6AAAAABEZI2IDSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGE2TANRRG4. You are receiving this because you authored the thread.

Lawrence37 commented 5 months ago

Thanks for the sample DNG!

There are a few ways to fix vignetting in RawTherapee. For the DJI Mini4 Pro, I think the best options are the manual correction (Transform tab > Lens / Geometry > Vignetting Correction) or flat-field correction (Raw tab > Flat-Field). Manual correction doesn't require any setup. Just "eyeball" the results. The disadvantage is that the correction will not be exact and does not fix any color cast which typical of these kinds of cameras. Flat-field correction gives precise results and also fixes color casts, but requires you to shoot flat-field images. RawPedia has an extensive article on using flat-fields in RawTherapee.

The JPEG looks ok because the vignetting has been corrected in-camera, but not the DNG. The other software you mention are also applying correction because they know the vignette parameters. A note for RawTherapee developers: The DNG contains a GainMap in OpcodeList3 which might be for vignette correction. RawTherapee supports the OpcodeList2 GainMap but not the OpcodeList3 one.

For profiled lens corrections, RawTherapee uses Lensfun. They don't have any data for the DJI Mini4 Pro. If you have the time and equipment, the Lensfun team accepts sample images (see https://lensfun.github.io/calibration/).

RAWimageViewer commented 5 months ago

L

And thank you for this very nice and detailed response!

I will take time to work with vignetting as you indicate (which I did a little before contacting you first, but now know to focus on this a bit more). The uneven coloration issue however remains a bit and that is an unpleasant aspect to trying to bring the entire photo to as even a presentation as one sees in the JPEG.

QUESTION: Do you think it possible if I contact the DJI tech team that they might be able to do something (maybe software wise in drone) to make vignetting adjustment for DNG?

If yes, I will contact them and provide an image and indirectly note I was given advisement about DNG and vignetting issue (without directly a reference to you … unless you give an Okay for being reference).

DJI customer service is rather good, so might be worth a try.

Might OpcodeList3 come in the future?

And regarding Lensfun, with my science tendency I would be glad to be helpful and could send them a couple different photos to work with. QUESTION: Would I be able to provide some background information and details along with a couple photo examples?

I will wait for answers to questions and then make one last step to help and thank you so much!

T

On Mar 17, 2024, at 3:03 PM, Lawrence37 @.***> wrote:

Thanks for the sample DNG!

There are a few ways to fix vignetting in RawTherapee. For the DJI Mini4 Pro, I think the best options are the manual correction (Transform tab > Lens / Geometry > Vignetting Correction) or flat-field correction (Raw tab > Flat-Field). Manual correction doesn't require any setup. Just "eyeball" the results. The disadvantage is that the correction will not be exact and does not fix any color cast which typical of these kinds of cameras. Flat-field correction gives precise results and also fixes color casts, but requires you to shoot flat-field images. RawPedia has an extensive article on using flat-fields in RawTherapee.

The JPEG looks ok because the vignetting has been corrected in-camera, but not the DNG. The other software you mention are also applying correction because they know the vignette parameters. A note for RawTherapee developers: The DNG contains a GainMap in OpcodeList3 which might be for vignette correction. RawTherapee supports the OpcodeList2 GainMap but not the OpcodeList3 one.

For profiled lens corrections, RawTherapee uses Lensfun. They don't have any data for the DJI Mini4 Pro. If you have the time and equipment, the Lensfun team accepts sample images (see https://lensfun.github.io/calibration/).

— Reply to this email directly, view it on GitHub https://github.com/Beep6581/RawTherapee/issues/6995#issuecomment-2002580926, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHARQ5QVDB3YFTMFQ5OGMLTYYXSIPAVCNFSM6AAAAABEZI2IDSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGU4DAOJSGY. You are receiving this because you authored the thread.

Lawrence37 commented 5 months ago

Most likely the DNGs already have the information required to fix the vignette. I referenced #6998 which I think will allow RawTherapee to use the vignette data.

For Lensfun, take some time to read through the link. There are a few scenarios that affect what information they need from you. All the instructions are available on that page or in the links on that page.

RAWimageViewer commented 5 months ago

Ok thanks.

For clarity for me … the reference number #6998 … does that mean the photos I shared and other information will help RAWTherapee to make accommodations and adjustments so my uploading DNG sometime in future will work better?

And yes, I’ll follow the instructions for Lensfun.

T

On Mar 17, 2024, at 4:01 PM, Lawrence37 @.***> wrote:

Most likely the DNGs already have the information required to fix the vignette. I referenced #6998 https://github.com/Beep6581/RawTherapee/issues/6998 which I think will allow RawTherapee to use the vignette data.

For Lensfun, take some time to read through the link. There are a few scenarios that affect what information they need from you. All the instructions are available on that page or in the links on that page.

— Reply to this email directly, view it on GitHub https://github.com/Beep6581/RawTherapee/issues/6995#issuecomment-2002599710, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHARQ5T325DTIWR74AWGKMLYYXZDLAVCNFSM6AAAAABEZI2IDSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGU4TSNZRGA. You are receiving this because you authored the thread.

Lawrence37 commented 5 months ago

Your sample image does help and it's possible we need you to upload it again, but I have downloaded it and can share it with other developers if needed. I recommend sharing a sample to raw.pixls.us.The RPU is a collection of raw images from all kinds of cameras. RawTherapee and other FOSS photo software use it to add or improve support for raw image formats.

RAWimageViewer commented 5 months ago

Hi Lawrence,

https://filebin.net/72ynf4yss0ov7lk7 I have taken and loaded two additional photos to help RAWTherapee developers. Again, the DNG is from the DJI mini 4 pro drone and vignetting is needed for the DNG photo. I also included the jpeg version taken at same time.

I have yet to upload anything to lensfun at github but will offer these photos now and see if they can help develop a lens correction. Will do that after sending this email.

Todd

On Mar 16, 2024, at 5:56 PM, Lawrence37 @.***> wrote:

Duplicate of #6868 https://github.com/Beep6581/RawTherapee/issues/6868.

What you are seeing is vignetting. It's especially pronounced in many small sensor cameras like drones and smartphones (in my experience). What are the "camera settings for some of the DJI drones" you are talking about? You can use filebin.net to share large files. GitHub accepts smaller files as long as the extension is supported (rename xyz.pp3 to xyz.pp3.txt or put them in a .zip file before uploading).

— Reply to this email directly, view it on GitHub https://github.com/Beep6581/RawTherapee/issues/6995#issuecomment-2002150617, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHARQ5RQVPBWOR4J4MWYJJDYYS5ZNAVCNFSM6AAAAABEZI2IDSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGE2TANRRG4. You are receiving this because you authored the thread.

RAWimageViewer commented 5 months ago

Hi, Just uploaded to lensfun and provided the exact set of photos I provided to you.

That would be two DNG photos with corresponding JPEG photos.

The first two photos I sent to you I think had a ND16/CPL filter on the drone camera. The second two photos had a UV filter … is I recall correctly.

Hope the lens profile can be created and shared with you!

Might vignetting adjustments be incorporated in RAWTherapee at any time? Just curious if that can be programmed into RT. I have to say doing corrections to the DJI mini 4 pro DNG photos is a bit discouraging because it takes a lot of effort to do the vignetting adjust and then the color around the peripheral area of the photos is still not optimal. But that may be me and I may get better at this in the future!

Thanks!

Todd

On Mar 18, 2024, at 1:48 AM, Lawrence37 @.***> wrote:

Your sample image does help and it's possible we need you to upload it again, but I have downloaded it and can share it with other developers if needed. I recommend sharing a sample to raw.pixls.us https://raw.pixls.us/.The RPU is a collection of raw images from all kinds of cameras. RawTherapee and other FOSS photo software use it to add or improve support for raw image formats.

— Reply to this email directly, view it on GitHub https://github.com/Beep6581/RawTherapee/issues/6995#issuecomment-2002978562, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHARQ5X6CNLWYAZPC7F5BRDYYZ53ZAVCNFSM6AAAAABEZI2IDSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSHE3TQNJWGI. You are receiving this because you authored the thread.

RAWimageViewer commented 3 months ago

Hi Lawrence,

It has been some time since we last had correspondence and I thought it nice to just add something today.

I just went with my DJI Mini 4 Pro drone to a location where I took four sets of photos.

The drone camera has 12 mega pixel and 48 mega pixel settings and so I took JPEG + RAW (DNG) sets / pairs of photos with both the 12 and 48 mega pixel settings and then used four different camera filters to add to that comparison.

I unloaded the photos and then accessed them in RAWTherapree and the vignetting problem (uneven exposure and color offset was rather obvious) showed up again in all photos comparisons when looking at the DNG photos. JPEGs looked nice. It is horrible and to try and offset the RAW photo outer difference from the inner photo … just too much to deal with.

HOWEVER, loading the same photos into darktable 4.6.1 … the DNG images have a more even exposure across the entire photo and not at all the same appearance as in RAWTherapee.

CONCLUSION: something differs between RAWTherapee and darktable. I did send example photos to you before but I had not spent time comparing RAWTherapee to darktable because the darktable program seemed unstable … but with a recent Apple OS update darktable seems better.

So … what is happening in RAWTherapee when a DJI Mini 4 Pro DNG RAW photo is imported?

I don’t know but I’m now going to spend time learning about darktable and until RAWTherapee is updated in some way … I won’t be able to use RAWTherapee … is there any update coming in the future?

Anyway … just thought I’d share that.

Take Care

Todd

On Mar 17, 2024, at 3:03 PM, Lawrence37 @.***> wrote:

Thanks for the sample DNG!

There are a few ways to fix vignetting in RawTherapee. For the DJI Mini4 Pro, I think the best options are the manual correction (Transform tab > Lens / Geometry > Vignetting Correction) or flat-field correction (Raw tab > Flat-Field). Manual correction doesn't require any setup. Just "eyeball" the results. The disadvantage is that the correction will not be exact and does not fix any color cast which typical of these kinds of cameras. Flat-field correction gives precise results and also fixes color casts, but requires you to shoot flat-field images. RawPedia has an extensive article on using flat-fields in RawTherapee.

The JPEG looks ok because the vignetting has been corrected in-camera, but not the DNG. The other software you mention are also applying correction because they know the vignette parameters. A note for RawTherapee developers: The DNG contains a GainMap in OpcodeList3 which might be for vignette correction. RawTherapee supports the OpcodeList2 GainMap but not the OpcodeList3 one.

For profiled lens corrections, RawTherapee uses Lensfun. They don't have any data for the DJI Mini4 Pro. If you have the time and equipment, the Lensfun team accepts sample images (see https://lensfun.github.io/calibration/).

— Reply to this email directly, view it on GitHub https://github.com/Beep6581/RawTherapee/issues/6995#issuecomment-2002580926, or unsubscribe https://github.com/notifications/unsubscribe-auth/BHARQ5QVDB3YFTMFQ5OGMLTYYXSIPAVCNFSM6AAAAABEZI2IDSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGU4DAOJSGY. You are receiving this because you authored the thread.

kmilos commented 3 months ago

something differs between RAWTherapee and darktable

darktable implemented support for DNG OpcodeList3 vignetting: https://github.com/darktable-org/darktable/pull/12880

Note that this is for the FixVignetteRadial opcode only rather than GainMap (also supported only for OpcodeList2).

sgotti commented 3 months ago

I'm in the work of porting the various embedded metadata lens corrections from darktable (ART already did this) to Rawtherapee. This also includes the dng opcode3 WarpRectilinear. I'm using the new Fuji algorithm that I developed for darktable in https://github.com/darktable-org/darktable/pull/14468 (ART uses the old version) and I'm also trying to port the Panasonic distortion corrections. This is probably related to issue #3254.

@Lawrence37 Let me know if I should open a new issue to track this work and future PRs.

Lawrence37 commented 3 months ago

@RAWimageViewer The best way to get updates on the progress is to track the linked issue (#6998).

@sgotti It's a good idea to create issues and self-assign them so people know you are working or planning to work on them before you have enough working code to create a pull request.