Beep6581 / RawTherapee

A powerful cross-platform raw photo processing program
https://rawtherapee.com
GNU General Public License v3.0
2.81k stars 316 forks source link

Panasonic GX9 color profile #4540

Closed dpawson closed 6 years ago

dpawson commented 6 years ago

https://filebin.net/ktdcj5hptq1x3x1t daylight image. I have the data files associated with this image if needed. Charge: R170112

Tungsten file added - 194

Beep6581 commented 6 years ago

@dpawson thank you but that shot does not meet the quality requirements. Please follow the shooting instructions at http://rawpedia.rawtherapee.com/How_to_create_DCP_color_profiles#Shooting_the_Color_Target and submit both daylight + tungsten shots.

dpawson commented 6 years ago

Tungsten added. What else is wrong? OK, very precise. Will try again later today.

dpawson commented 6 years ago

Try again https://filebin.net/ktdcj5hptq1x3x1t

Wolf Faust C1 Din A4, IT8.7 Supporting files available

Beep6581 commented 6 years ago

@dpawson take 2 is much better, but it seems the shot marked as tungsten is actually also daylight.

The shot is now evenly lit, not distorted, fills not more and not less than ~1/3rd of the frame, has a non-reflective background, in-focus, low ISO, good aperture. All good.

Remaining areas of improvement:

  1. I could increase exposure by +1.3EV without clipping. Would be perfect if you could bracket by 1EV and send the 0EV and +1EV shots.
  2. Please include info on where you took the daylight shot, as daylight in Norway is a bit different to daylight in Uganda, for example, and I'd like to keep track of this info for future reference.
Beep6581 commented 6 years ago

camconst.json so far, white levels need to be improved:

diff --git a/rtengine/camconst.json b/rtengine/camconst.json
index 35cb524a..d30bd7a2 100644
--- a/rtengine/camconst.json
+++ b/rtengine/camconst.json
@@ -1938,6 +1938,15 @@ Camera constants:
         }
     },

+    { // Quality X, no white frames yet
+        "make_model": [ "Panasonic DC-GX9" ],
+        "dcraw_matrix": [ 7564,-2263,-606,-3148,11239,2177,-540,1435,4853 ], // ColorMatrix2 from Adobe DNG Converter 10.3
+        "ranges": {
+            "black": 16, // 271? BlackLevelRed=128 + BlackLevel=143 143 143 143
+            "white": 4080
+        }
+    },
+
     { // Quality B, Same as Panasonic G7
         "make_model": [ "Panasonic DMC-G8", "Panasonic DMC-G80", "Panasonic DMC-G81", "Panasonic DMC-G85" ],
         "dcraw_matrix": [ 7610,-2780,-576,-4614,12195,2733,-1375,2393,6490 ], // DNG_v9.7 D65
dpawson commented 6 years ago

tungsten.0 and tungsten.1 (+1ev) uploaded, same location.

Beep6581 commented 6 years ago

@dpawson in addition to the color target shots, we also need to find out where the sensor clips so that clipped white stays white instead of getting a magenta cast. Maybe this data can be copied from some other Panasonic model - @iliasg is the expert.

If you would like to go ahead and take the required shots to measure the white levels, the instructions are here: http://rawpedia.rawtherapee.com/Adding_Support_for_New_Raw_Formats Looking at the other Panasonics, it looks like aperture scaling isn't taking place so we don't need those, we'd only need white-frame shots at: ISO 100, 125, 160, 200, 400, 800, 1600, 3200, 6400, 12800, 25600.

Beep6581 commented 6 years ago

@dpawson there is too much glare in the tungsten shot (over 1.5 stops) and a clear light fall-off from the top-right corner to the bottom-left: screenshot_20180507_154836

dpawson commented 6 years ago

"white frame"? What is meant by that? Under daylight and tungsten?

The 'fall off' is due to my avoiding glare. Suggestions please how to set up to avoid fall-off and glare?

Sorry, you asked for location. 52.5695° N, 0.2405° W - mid UK

Beep6581 commented 6 years ago

Under daylight and tungsten?

The light used for white-frames is irrelevant, as long as the entire frame is clipped white.

What is meant by that?

See the link.

Suggestions please how to set up to avoid fall-off and glare?

First, position yourself and the light source in such a way that there is no glare. Then, if due to this the light fall-off becomes too strong, move the light source further away but keep the same angle. I usually have the lamp 2 meters behind me, at an angle of 10-30 degrees.

P.S. Avoiding glare is only relevant for the target shot. The white-frame shots are meant to be entirely clipped, so it's as simple as setting your camera to each required ISO, e.g. ISO120, pointing at the sky or at a lamp and overexposing by several stops so that the entire frame is completely clipped.

dpawson commented 6 years ago

I was unable to find a definition of 'white' in the link? Can I use the reverse of the ref card?

10-30 degrees gives me glare, hence the movement. I'll try, but no promises. IMHO daylight would be easier than a tiny tungsten. I'll wait until tomorrow.

Beep6581 commented 6 years ago

I was unable to find a definition of 'white' in the link? Can I use the reverse of the ref card?

I was referring to this link: http://rawpedia.rawtherapee.com/Adding_Support_for_New_Raw_Formats But let's leave that for now until the target shots are done. Maybe in the meanwhile @iliasg might decide that the white-frames are necessary.

IMHO daylight would be easier than a tiny tungsten.

What do you mean?

dpawson commented 6 years ago

Note the white to black line at the bottom of the target?

On 7 May 2018 at 15:24, Beep6581 notifications@github.com wrote:

I was unable to find a definition of 'white' in the link? Can I use the reverse of the ref card?

I was referring to this link: http://rawpedia.rawtherapee.com/Adding_Support_for_New_Raw_Formats But let's leave that for now until the target shots are done. Maybe in the meanwhile @iliasg https://github.com/iliasg might decide that the white-frames are necessary.

IMHO daylight would be easier than a tiny tungsten.

What do you mean?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Beep6581/RawTherapee/issues/4540#issuecomment-387081530, or mute the thread https://github.com/notifications/unsubscribe-auth/AASOJ8B5JHUrzOd-cDMGbUNZxFnK9bMFks5twFkvgaJpZM4T0zI9 .

-- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ.

dpawson commented 6 years ago

On 7 May 2018 at 15:24, Beep6581 notifications@github.com wrote:

IMHO daylight would be easier than a tiny tungsten.

What do you mean?

​Since Tungsten is a 'point light', whereas daylight is wider? Less reflections, less degradation.

Regards Dave​

-- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ.

dpawson commented 6 years ago

Two sets of full range of wb uploaded. wb1 to wb16, first half is inc wb, second decreasing. I suspect first half may not be sufficiently illuminated? Taken under Tungsten light.

iliasg commented 6 years ago

@dpawson, because GX9 looks a bit different from other Panasonics .. in order to have better support in RT .. Please, if possible, provide two sets of white frames at ALL ISO settings available as described in 1) and 2) at http://rawpedia.rawtherapee.com/Adding_Support_for_New_Raw_Formats .. "Needed Photos"

An easy way is to make your full display area bright white, then shoot with a lens which has no (or weak ) vignette from close distance so that all captured frame is white. Better is the shots to be totally out of focus. Use strong overexposure i.e. if auto exposure gives X values use +5EV compensation.

Additionally .. usefull would be to have two sets at all available ISOs of Black frames at all available ISOs .. i.e in dark environment, with lens cover put on, shoot at very close apperture (f22.0 ?) and high speed .. both with Long Exposure noise reduction on and off ..

dpawson commented 6 years ago

You have two sets there iliasg? Either describe what you want more clearly or expect something else.

http://rawpedia.rawtherapee.com/Adding_Support_for_New_Raw_Formats needs a good overhaul if you want 'amateurs' to help you with new cameras.

Beep6581 commented 6 years ago

needs a good overhaul

@dpawson the article seems clear to me, but if something is unclear to you, you're welcome to list specific improvements and to point out specific issues.

Quoting from the article:

Each photo must be completely overexposed everywhere! Do this by pointing your camera at a bright light (e.g. the sky, a lamp), zooming in as needed, and increasing exposure time until everything is absolutely clipped.

How can I make that more clear?

Here is an example of what is needed: https://filebin.net/hz77wksih4jhwc1n/DSC07209.zip?t=nf3kyyeu

Turn Long Shutter Noise Reduction off, take one shot like that at every ISO value at whatever exposure time is needed to clip everything in the frame but less then 1/2s, so let's say 1/20s. That's about 28 shots in total, and each shot must be completely clipped white, like my example. Next, turn Long Shutter Noise Reduction on, take another series of shots at every ISO value but this time make sure the exposure time for each shot is no less than 2 seconds. That's another ~28 shots. There will be a total of about 56 white-frame shots.

For the black frames, put the lens cap on to block all light, set the exposure time to something like 1/5000s, turn LSNR off, and take one shot at every ISO value. Each shot must be completely black. Then turn LSNR on and repeat. That's a total of about 56 black-frame shots.

Zip them up, upload, done.

dpawson commented 6 years ago

I will try and re-write the b/w section from my viewpoint (knowing nothing of all this).

IMHO - leave all post processing to another section. List what to do as a sequence. Colour, White, Black. Prep, exposure constraints, cautions. (e.g. is it possible to test for light drop off top L to bottom R?) This to stop an enthusiast having to repeat everything as I am.

Your assumption for example

Each photo must be completely overexposed everywhere! Do this by pointing your camera at a bright light (e.g. the sky, a lamp), zooming in as needed, and increasing exposure time until everything is absolutely clipped.

I ask. How much over exposed? What if my light isn't very bright? Point at the sky? Do you want white? Is a red light OK? Any type of white light / lamp? Clear blue sky? Zoom in? Should I defocuss on purpose? How do I know when an image is clipped?

IMHO you are viewing this as an expert, not as a newbie trying to help?

I will try - then you criticise my writings!

Proposing to use a CN-160, bank of LED lamps as target. Defocussed, 1/10 sec, f8, through all ISO. Unless something wrong there?

regards

On 7 May 2018 at 18:52, Beep6581 notifications@github.com wrote:

needs a good overhaul

@dpawson https://github.com/dpawson the article seems clear to me, but if something is unclear to you, you're welcome to list specific improvements and to point out specific issues.

Quoting from the article:

Each photo must be completely overexposed everywhere! Do this by pointing your camera at a bright light (e.g. the sky, a lamp), zooming in as needed, and increasing exposure time until everything is absolutely clipped.

How can I make that more clear?

Here is an example of what is needed: https://filebin.net/hz77wksih4jhwc1n/DSC07209.zip?t=nf3kyyeu

Turn Long Shutter Noise Reduction http://eng-ca.faq.panasonic.com/app/answers/detail/a_id/38836/%7E/how-do-i-set-the-long-shutter-noise-reduction%3F off, take one shot like that at every ISO value at whatever exposure time is needed to clip everything in the frame but less then 1/2s, so let's say 1/20s. That's about 28 shots in total, and each shot must be completely clipped white, like my example. Next, turn Long Shutter Noise Reduction on, take another series of shots at every ISO value but this time make sure the exposure time for each shot is no less than 2 seconds. That's another ~28 shots. There will be a total of about 56 white-frame shots.

For the black frames, put the lens cap on to block all light, set the exposure time to something like 1/5000s, turn LSNR off, and take one shot at every ISO value. Each shot must be completely black. Then turn LSNR on and repeat. That's a total of about 56 black-frame shots.

Zip them up, upload, done.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Beep6581/RawTherapee/issues/4540#issuecomment-387147798, or mute the thread https://github.com/notifications/unsubscribe-auth/AASOJ-fOU5pfAO_xFP8O6A-lpyhH5iUEks5twInHgaJpZM4T0zI9 .

-- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ.

Floessie commented 6 years ago

I ask.

Yes, please.

How much over exposed?

Completely.

What if my light isn't very bright?

Take your time.

Point at the sky?

Only when there's daylight.

Do you want white?

As stated.

Is a red light OK?

No.

Any type of white light / lamp?

You can also use a red, a green, and a blue lamp at once.

Clear blue sky?

Not necessarily.

Zoom in?

As needed.

Should I defocuss on purpose?

Yes.

How do I know when an image is clipped?

It's all white.

You're asking funny questions. The two sentences you quoted above have all the answers (maybe except for the defocusing).

IMHO you are viewing this as an expert, not as a newbie trying to help?

Don't know how @Beep6581 is viewing this, but I think anyone who's ever taken a photo in sunlight should know what "overexposed" means. Now add "completely", and it seems like 1 + 1 to me. Why make a thousand words, if two concise sentences will do? :confused:

dpawson commented 6 years ago

On 7 May 2018 at 20:02, Floessie notifications@github.com wrote:What if my light isn't very bright?

You're asking funny questions. The two sentences you quoted above have all the answers (maybe except for the defocusing).

​Maybe they do for you. Not for me. Not knowing the destination, I must question each step of the journey?​

IMHO you are viewing this as an expert, not as a newbie trying to help?

Don't know how @Beep6581 https://github.com/Beep6581 is viewing this, but I think anyone who's ever taken a photo in sunlight should know what "overexposed" means. Now add "completely", and it seems like 1 + 1 to me. Why make a thousand words, if two concise sentences will do? 😕

​If it would (and was clear to all) then I would agree.

As I say, you view the instructions having done the procedure and understand it? I see it as instructions to be followed blindly, hence my questions of detail

Dave​

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Beep6581/RawTherapee/issues/4540#issuecomment-387168968, or mute the thread https://github.com/notifications/unsubscribe-auth/AASOJ2GrjOfnkVCAhaBiA8bk6vpce7nBks5twJpYgaJpZM4T0zI9 .

-- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ.

Beep6581 commented 6 years ago

@dpawson how are the tungsten shots of the color target coming along? To avoid confusion, this issue is only about shots of the color target. We can continue the white-frame and black-frame discussion in a new issue.

dpawson commented 6 years ago

On 10 May 2018 at 10:06, Beep6581 notifications@github.com wrote:

@dpawson https://github.com/dpawson how are the tungsten shots of the color target coming along?

​Done and posted ​

To avoid confusion, this issue is only about shots of the color target. We can continue the white-frame and black-frame discussion in a new issue.

​Have you looked at the data that comes with the Wolf Faust stuff? Covers black to white - bottom row. I think the data gives true colour values for that range, as well as the colours?

Dave​

-- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ.

dpawson commented 6 years ago

From the target I bought. If any of these files are of use, please say.

R170112.eps and R170112.hist provide the spectral data of the target. The files can be used with the X-Rite ColorShop V2.x and ColorShop X software. A full functional but time limited demo version of the software can be downloaded from the X-Rite site at http://www.x-rite.com. Note that the ColorShop file only contains the spectral data from 390-700nm or 400-700nm while the normal reference file is based on 380-780nm data. Tests done show that this limited precission by ColorShop causes only minor faults. Additional spectral file formats are available on request.

The "Extras" directory contains files for illuminants other than D50. It also contains data for the CIE 1964 10 degree observer. Color data for even more illuminants or observers can be calculated using the spectral data (ColorShop can easily do this for you).

Following illuminants/observer files are available in "Extras":

R170112A.txt - Illuminant A, 2 Degree Observer, Illuminant A was intended to represent the illumination from incandescent/tungsten light sources (approx. 2856K) used in general lighting and in projectors. R170112A10.txt - Same as R170112A.txt but 10 degree observer

R170112C.txt - Illuminant C, 2 Degree Observer, Represents average daylight with correlated colour temperature of about 6800K. R170112C10.txt - Same as R170112C.txt but 10 degree observer

R170112D5010.txt - Same as R170112.txt but 10 degree observer

R170112D55.txt - Illuminant D55, 2 Degree Observer, Representing a phase of daylight with a correlated color of approximately 5503K (mid-morning or mid-afternoon daylight) R170112D5510.txt - Same as R170112D55.txt but 10 degree observer

R170112D65.txt - Illuminant D65, 2 Degree Observer, Representing a phase of daylight with a correlated color of approximately 6504K. (Average northern daylight) R170112D6510.txt - Same as R170112D65.txt but 10 degree observer

R170112D75.txt - Illuminant D75, 2 Degree Observer, Representing a phase of daylight with a correlated color of approximately 7504K (North sky or overcast daylight ) R170112D7510.txt - Same as R170112D75.txt but 10 degree observer

R170112F2.txt - Illuminant F2, 2 Degree Observer, Represents a cool white fluorescent lamp with a correlated color temperature of 4200K. R170112F210.txt - Same as R170112F2.txt but 10 degree observer

R170112F7.txt - Illuminant F7, 2 Degree Observer, Represents a broad-band (continuous-spectrum) fluorescent daylight lamp with a correlated color temperature of 6500K. R170112F710.txt - Same as R170112F10.txt but 10 degree observer

R170112F11.txt - Illuminant F11, 2 Degree Observer, Represents a narrow-band (line-spectrum) white fluorescent lamp with a correlated color temperature of 4000K. R170112F1110.txt - Same as R170112F2.txt but 10 degree observer

On 10 May 2018 at 10:06, Beep6581 notifications@github.com wrote:

@dpawson https://github.com/dpawson how are the tungsten shots of the color target coming along? To avoid confusion, this issue is only about shots of the color target. We can continue the white-frame and black-frame discussion in a new issue.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Beep6581/RawTherapee/issues/4540#issuecomment-387997722, or mute the thread https://github.com/notifications/unsubscribe-auth/AASOJ9MkocXOEnU-q5UjKHFuJd-ksEAMks5txAMLgaJpZM4T0zI9 .

-- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ.

Beep6581 commented 6 years ago

@dpawson

Done and posted ​

Where? It's difficult making sense of your replies, as things get jumbled when you reply by email.

dpawson commented 6 years ago

On 10 May 2018 at 18:16, Beep6581 notifications@github.com wrote:

@dpawson https://github.com/dpawson

Done and posted ​

Where? It's difficult making sense of your replies, as things get jumbled when you reply by email.

​Same place you said to use?

https://filebin.net/ktdcj5hptq1x3x1t

Dave​

-- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ.

Beep6581 commented 6 years ago

@dpawson once more, the shot labeled "tungsten" is actually daylight: https://filebin.net/ktdcj5hptq1x3x1t/Panasonic.GX9.tungsten.rw2?t=tsc2hsaf

Please upload a shot of the target taken using a tungsten incandescent light bulb, with the same framing as your latest shots but making sure that there is no glare, and making sure that it was actually taken using a tungsten light bulb. You can upload to this new bin so that we don't mix the new shots up with the glared and mislabeled older shots: https://filebin.net/owgvkcgwworevdi8

dpawson commented 6 years ago

My interpretation of the 'instructions' needed by a new camera owner to help RT. Markdown. I can find no requirement to take 'black' levels.

mkprofile.zip

dpawson commented 6 years ago

https://filebin.net/owgvkcgwworevdi8 gx9.zip What I understand to be a full set of GX9 images daylight, tungsten (colour) and white over the ISO range of the camera.

If these are wrong, please find someone who understands your instructions better than I.

Dave

On 12 May 2018 at 10:17, Beep6581 notifications@github.com wrote:

@dpawson https://github.com/dpawson once more, the shot labeled "tungsten" is actually daylight: Panasonic.GX9.tungsten.rw2

Please upload a shot of the target taken using a tungsten incandescent light bulb, with the same framing as your latest shots but making sure that there is no glare, and making sure that it was actually taken using a tungsten light bulb. You can upload to this new bin so that we don't mix the new shots up with the glared and mislabeled older shots: https://filebin.net/owgvkcgwworevdi8

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Beep6581/RawTherapee/issues/4540#issuecomment-388542119, or mute the thread https://github.com/notifications/unsubscribe-auth/AASOJ3IJBqNvs9XoqFLF5L0D-8VkuEJ1ks5txqiVgaJpZM4T0zI9 .

-- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ.

Beep6581 commented 6 years ago

The tungsten shots are all 2.7EV underexposed. They look fine otherwise.

Beep6581 commented 6 years ago

DCP added in 8b235464.

Beep6581 commented 6 years ago

I rewrote http://rawpedia.rawtherapee.com/Adding_Support_for_New_Raw_Formats