Panasonic DC-FZ10002: WB presets, noise profiles and color profile (Question)

[666Shadowlord666]

I am a user of a Panasonic DC-FZ10002, which has received base support with Darktable 3.2.1. However, there are no WB presets or noise profiles available. I am considering creating both, however, I am unsure if this is even necessary:

The FZ1000 already offers both WB presets and noise profiles in Darktable and this camera should have an identical sensor and lens.

Can the existing data simply be used for the FZ10002? Or do these have to be created completely from scratch?

Something similar already seems to work for lens correction: The FZ10002 is not listed on Lensfun, but in Darktable the profile of the FZ1000 is automatically applied for lens correction.

I also have another question: If I wanted to create a color profile for the FZ10002, then a SpyderCHECKR 24 would be sufficient to do this. Is that correct? Would an IT8 target (from Wolf Faust) give visibly better results? I could rent a SpyderCHECKR 24 for €6 a day or buy an IT8 target for €35.

[johnny-bit]

if the sensor is identical (which can be checked with comparing values in adobe_coeff.c) then i guess that wb and noise can simply be copied over.

creating new color response profiles is not recommended. modern colour flow using color calibration module allows for better and more robust camera-independed way of handing colours.

[666Shadowlord666]

The values for the FZ10002 and FZ1000 do not look identical in adobe_coeff.c. So the wb and noise cannot simply be copied over? Or do I understand this wrong?

{ "Panasonic DMC-FZ1000", { 7830,-2696,-763,-3325,11667,1866,-641,1712,4824 } },
{ "Panasonic DC-FZ10002", { 9803,-4185,-992,-4066,12578,1628,-838,1824,5288 } },

https://github.com/darktable-org/darktable/blob/master/src/external/adobe_coeff.c#L531

creating new color response profiles is not recommended.

OK, I did not know that. Then I will use the camera independent functions for this.

[johnny-bit]

Then it's different sensor and yo have to create new wb and noise data.

[github-actions[bot]]

This issue did not get any activity in the past 30 days and will be closed in 365 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

[gaetrua]

How can i help to create new WB and noise data for this camera?

[johnny-bit]

Hi @gaetrua :)

Follow https://github.com/darktable-org/darktable/wiki/Camera-support :)

That means:

1. For WB follow https://github.com/darktable-org/darktable/wiki/White-balance-presets
2. For noise profiles follow https://pixls.us/articles/how-to-create-camera-noise-profiles-for-darktable/ and then create new noise profile issue

[johnny-bit]

@666Shadowlord666 and @gaetrua - Have you managed to create wb presets and/or noise data? I'm rounding up remaining wb prests and this is the last issue. If you wish to have WB presets for DC-FZ10002 in release 3.6 you better hurry ;)

[666Shadowlord666]

Well, I had created the required photos for the white balance profile and noise profile for the camera, but then failed at the further steps. I think it is because I'm on Windows 10.

The tutorial says: "Execute: $ /usr/share/darktable/tools/extract_wb_from_images.sh *"

At first, I did not have the extract_wb_from_images.sh file at all. It is not included in the Windows version of darktable. Gladly, I found the file online.

Then I had problems to execute that SH file under Windows. I have now succeeded in doing so (using Windows Subsystem for Linux), but I only get a tiny file called darktable-whitebalance-20210615.tar.gz, which does not seem to contain any data.

I assume the script does not work under Windows without further ado? In the script I read something about "exiftool". Maybe the script does not find exiftool. However, I do not get a corresponding error message.

I would be very grateful for a tip on how to extract the required data.

The steps for creating noise profiles from my photos seem even more complicated or even impossible when using Windows. I can share these photos, maybe someone is willing to extract the data?

https://mega.nz/file/DXgQxKKJ#GjQSx3z8ViNCY0R28mcCbVc6HKn5kwsNXJQ6b0D6IqQ

Also, I'm not sure whether these photos are good enough. If they aren't, just tell me how I can improve them and I will do my best to shoot better ones.

[johnny-bit]

Unfortunately i think both wb and noise tools don't really work on windows :/

if the link you've shared also contains wb files I can extract the wb data.

@rawfiner can you take a look at attached files and see if you can generate noise data?

[johnny-bit]

@666Shadowlord666 - I see that the data linked does not contain WB shots. If you could send the package with WB hots I could do the wb profiles.

As for noise data - I tried them and get following error from the script:

===> Checking profiling RAW images correctness + Jpeg export
--> ISO 125:
    ./Noise_FZ10002/125_ISO.RW2
"./Noise_FZ10002/125_ISO.RW2" not over-exposed (0)
--> ISO 200:
    ./Noise_FZ10002/200_ISO.RW2
"./Noise_FZ10002/200_ISO.RW2" not over-exposed (0)
--> ISO 400:
    ./Noise_FZ10002/400_ISO.RW2
"./Noise_FZ10002/400_ISO.RW2" not over-exposed (0)
--> ISO 800:
    ./Noise_FZ10002/800_ISO.RW2
"./Noise_FZ10002/800_ISO.RW2" not over-exposed (0)
--> ISO 1600:
    ./Noise_FZ10002/1600_ISO.RW2
"./Noise_FZ10002/1600_ISO.RW2" not over-exposed (0)
--> ISO 3200:
    ./Noise_FZ10002/3200_ISO.RW2
"./Noise_FZ10002/3200_ISO.RW2" not over-exposed (0)
--> ISO 6400:
    ./Noise_FZ10002/6400_ISO.RW2
"./Noise_FZ10002/6400_ISO.RW2" not over-exposed (0)
--> ISO 12800:
    ./Noise_FZ10002/12800_ISO.RW2
"./Noise_FZ10002/12800_ISO.RW2" not over-exposed (0)

For some reason, on all shots, the bright spot is not overexposed, especially in blues. I wonder if something could be done about that...

[666Shadowlord666]

I've installed a Linux distro in Virtualbox and succeeded in creating the white balance data for the Panasonic DC-FZ10002. I hope the file contains all required data, otherwise I can try again. I have submitted the file via white balance issue.

https://github.com/darktable-org/darktable/issues/9248

I can try to make another set of photos for noise data later today, maybe they will turn out better.

[johnny-bit]

For noise people recommend shooting unfocused photos of a white wall with bright bulb on one end and dark patch on another... the method in the article might not give the best results when not setted up perfectly... I'll check white balance issue in a bit, thanks!

[rawfiner]

Sometimes the overexposure error is a bit too much sensitive. Most important is to have blurry images, with widespread histogram especially near black. You can run the script with -C option (IIRC) to avoid these checks. I am sorry I have a very busy month, I won't be able to run the script on the image shortly... Thanks for having tried it @johnny-bit ! I can be more reactive to check the generated profile though

[johnny-bit]

OK, I tried the script with -C option :) Can you take a look?

darktable-noiseprofile-20210616.tar.gz

[rawfiner]

The profile is very good. One point though: the FZ1000 has 1/3 of stops for the ISO, I assume it is the same for the FZ10002. We would need all possible ISO values please @666Shadowlord666

[johnny-bit]

@rawfiner - I've tried to look for it and reviews say that the camera has "8 iso settings from 125 to 12800"... however... According to manual https://www.panasonic.com/content/dam/Panasonic/au/en/support_manual/lumix-cameras/DC-FZ10002_advanced_operating-instructions.pdf (page 109) iso can go from 80/100 up to 25600 when extended iso is enabled...

so either "it can go as is" or @666Shadowlord666 - can you add shots with extended iso?

[rawfiner]

Ok, I'll let @666Shadowlord666 tell us if there are intermediate stop as well. We need extended iso shot as well by the way

[666Shadowlord666]

Oh, sorry, I forgot about that extended ISO function. There's also a function for intermediate stops (1/3), which I wasn't even aware of. I'll make a new set of photos right now.

[rawfiner]

Thanks!

[666Shadowlord666]

Next try. Full set with extended iso and intermediate stops (1/3). I hope these photos are good enough. https://mega.nz/file/jLYlWCKS#Ndm45V2PoWeDbCPiBxFA-kaSuhltSgFcs6etYq-bSUQ

[johnny-bit]

Hey!

Created noise profile from them. Can you check @rawfiner ?

darktable-noiseprofile-20210616.tar.gz

[rawfiner]

Hi, thanks ISO 80 should be done again please, profile fitting failed for this value others are fine, would be a bit better though if histogram would be a bit more widespread (maybe by overexposing a bit more)

[666Shadowlord666]

Here's a new set of photos. I have tried to overexpose a bit more and to get a more widespread histogram. Hopefully, I did not overexpose too much.

https://mega.nz/file/XTYHRQCL#gw8RGP5I5TYnVlOQpyyrZ00j_fwQKcYOQ8kq5JQ4q04

Because the photos for ISO 80, 16000, 20000 and 25600 seem a bit too bright, I've added additional versions for them in a subfolder.

[johnny-bit]

Actually the ISO 80 ones aren't bright enough I think

Here's the result for your set: darktable-noiseprofile-20210617.tar.gz

and here's for alt: darktable-noiseprofile-20210617_alt.tar.gz

can you try and do just ISO 80, but make it SUPER bright especially in overexposed part?

[666Shadowlord666]

Here are a few more shots, only ISO 80.

https://mega.nz/file/ODhHkATQ#hK4uBe7aUxAPdVzM9cCp4IFEMr0vJUq_JcWEOjogxco

[johnny-bit]

Quite fascinating really:

darktable-noiseprofile-20210617_alliso80.tar.gz

@rawfiner - judging by how iso80 behaves I think the set from https://github.com/darktable-org/darktable/issues/8537#issuecomment-863303258 would work best...

[rawfiner]

I agree with you @johnny-bit , this set is the best:

Here's the result for your set: darktable-noiseprofile-20210617.tar.gz

better histograms than the first one. All values are nice except ISO 80: at this point I propose that we correct it manually. In the noise profiles, the "a" values behave very regularly. "a" from ISO 100 is basically half "a" from ISO 200. I propose to use this "a" values for green channel for ISO 80 (half the one of ISO 160): 3.56368746385176e-06 b can be left unchanged, as well as "a" from red and blue (which are very close to half ISO 160 as expected)