Wrong colours with Pentax PEF files and latest K-r template

[docjok]

LightZone v3.9.2 running on Mac OS X 10.6.8 and Java 1.6.0_24 dcraw v9.11 and latest K-r template

Problem: image colours shown in the edit window are not correct. It's hard to say, but maybe red and blue channels are swapped and there is a strong green tint. The same problem occurred prior to installing the new template. The PEF file can be viewed fine in Preview and the embedded preview in the PEF file is displayed correctly in LZ's browse display.

Can files be attached in these issues?

[Doug-Pardee]

That will be something to do with the dcraw program. The dcraw 8.99 shipped with the "factory" LightZone package produces those colors. The updated dcraw, 9.11, produces proper colors on Windows.

This is probably something specific with getting dcraw running on a Mac. Which, unfortunately, I know nought about. A friend of mine compiled the dcraw package for Mac, and had no way of testing it. I'm not even positive on where it gets installed.

Let me see if I can get one of our Mac users to help out on this.

[docjok]

I tried running the 9.11 dcraw app I downloaded from here on the command line and passing it my same test file. Using just the default options, the output image was correct in terms of the colours. However, I think you said somewhere that dcraw is only used for some of the RAW conversion by LZ. Are the command-line options passed to dcraw known or in any log file?

Is dcraw for Mac built in an X-Code project?

[Doug-Pardee]

For the Edit window, LZ uses dcraw -v -i whatever.PEF to get the file information. Try that yourself with a K-r Raw file, and toward the end the new (9.11) LZ version should include this data:

Raw colors: 3
Filter pattern: BGGRBGGRBGGRBGGR
Camera RGB Profile: 0.828590 0.136128 0.035283 0.041664 1.229485 -0.271149 0.020067 -0.241328 1.221261
Daylight multipliers: 1.735330 0.999954 1.506451
Camera multipliers: 14976.000000 8192.000000 19136.000000 8192.000000

The "Filter Pattern" is the biggest culprit. The pattern on the K-r is different from the default (shown below).

Also, the "Camera RGB Profile" is fairly important. That line only appears in the LZ version, and the coefficients on it are used to transform from "camera raw" color space to LightZone's 16-bit linear color space (which, by the way, has the same gamut and white point as ProPhoto RGB).

Here's the bad output from the dcraw 8.99 included in the 3.9 version. Note both the Filter Pattern and the Camera RGB Profile:

Raw colors: 3
Filter pattern: RGGBRGGBRGGBRGGB
Camera RGB Profile: 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000
Daylight multipliers: 1.000000 1.000000 1.000000
Camera multipliers: 14976.000000 8192.000000 19136.000000 8192.000000

You'll notice that the R and B channels are swapped, which is something you suspected, and the red and blue daylight multipliers are much lower while the green multiplier is almost the same. The daylight multipliers are derived from the RGB profile.

I have no idea how the current Mac version of dcraw was built. A friend did it for me, so you're probably the guinea pig. :-( If you know how to build from C code, the source is here on the github project, and in the Windows and Linux directories the README files tell what options were used for compiling those versions.

[docjok]

Thanks for the info.

Using the new 9.11 dcraw version, here is the output I got from a PEF file.

Filter pattern: BGGRBGGRBGGRBGGR
Camera RGB Profile: 0.828590 0.136128 0.035283 0.041664 1.229485 -0.271149 0.020067 -0.241328 1.221261
Daylight multipliers: 1.735330 0.999954 1.506451
Camera multipliers: 15520.000000 8192.000000 12416.000000 8192.000000

This is pretty much as you listed, except for two of the Camera Multipliers.

I just need to get up to speed with git and GitHub, first, and I'll see if there are any clues in the source code. I've only used SVN before.

I tried renaming the dcraw in the LightZone Java directory just to confirm that it was running the new version, which it was.

[krisw]

Doug-Pardee asked me to test on my Mac how Lightzone 3.9.2 shows IMGP1614.PEF in Edit mode.

In BROWSE the image looks good: the Kodak Grey Scale merges at steps18 and 19, step A is not as bright a white as the the foreground at the bottom of the image, but otherwise colours look OK.

In EDIT the image has a cast. The colours are dulled by the cast but not overtaken by it, so blues, reds, greens, yellows are distinct but muted. Blacks and greys are muted, grey steps moving towards white develop a green tinge and white is a pale green. The bright green object bottom left remains visibly green but is not the bright green of the image as seen in BROWSE.

To my eyes the cast is certainly there but it is not the brilliant green described in Doug's email, it is more a pale green tinged with blue. My wife described it as a bluey green.

I hope this helps.

[docjok]

Thanks krisw,

The main problem I experience is that the R and B channels are swapped. Until they are the right way around, it is hard to comment on the green cast, but it certainly looks like one is there. By the sounds of it, you do not see this channel swap, which is interesting.

What I have picked up is that LZ uses dcraw to interrogate the RAW file and determine the RAW data parameters, and then uses these to do its own demosaic. Is that right? If I run dcraw explicitly to convert to RGB, the output is fine. That would tend to imply that LZ is either passing dcraw an option which breaks something, or LZ is not correctly processing things internally. But the fact that LZ works on your system has kinda thrown a cat amongst pigeons.

I guess the other things are your system. Are you running Lion or Snow Leopard? And what Java version?

Is IMGP1614.PEF available for download?

Thanks for your time and efforts.

[docjok]

Well. I'm confused now. I took a couple more shots to better understand what is going on. I have been shooting DNG. The latest shots look fine in LZ's edit mode, however the original shot still does not. Aside from shooting parameters, the shots are the same. I'll see if I can work out why, but there must be something in the file that LZ did not like.

Thanks for your help so far.

[krisw]

hi, docjok,

Doug has sent me an email telling me that he has put me down as an official collaborator on LightZombie github. I have replied to him telling him that I lack the technical knowledge to play such a role in respect of the inner workings of any program. I am a computer user for my photo hobby and interest, nothing more.

I have a little knowledge of aspects of Mac OS X based on personal experience and enquiry, but that is all. I am willing to pass on what I know as an issue arises. The workings of dcraw, channel swapping, etc. are outside my sphere at present, though I may gradually begin to understand something of it as time passes. I am sorry if this sounds negative, but I remain willing to test any development that Doug brings forward. He has asked me to try a replacement curve for the Canon 7D which I have yet to do.

I am unable to comment on your reference to channel swapping and LZ's use of draw, etc. It is all beyond my knowledge at present.

With regard to your questions:

I am running Lion version 10.7.2 with the latest updates including the new Java release - Java 1.6.0_29.

IMGP1614.PEF is available for download from the DPReview website review of the Pentax K-r. Go to http://movies.dpreview.com.s3.amazonaws.com/pentax_kr/IMGP1614.PEF.zip

Regards krisw

On 23 Nov 2011, at 02:32, docjok wrote:

Thanks krisw,

The main problem I experience is that the R and B channels are swapped. Until they are the right way around, it is hard to comment on the green cast, but it certainly looks like one is there. By the sounds of it, you do not see this channel swap, which is interesting.

What I have picked up is that LZ uses dcraw to interrogate the RAW file and determine the RAW data parameters, and then uses these to do its own demosaic. Is that right? If I run dcraw explicitly to convert to RGB, the output is fine. That would tend to imply that LZ is either passing dcraw an option which breaks something, or LZ is not correctly processing things internally. But the fact that LZ works on your system has kinda thrown a cat amongst pigeons.

I guess the other things are your system. Are you running Lion or Snow Leopard? And what Java version?

Is IMGP1614.PEF available for download?

Thanks for your time and efforts.

---

Reply to this email directly or view it on GitHub: https://github.com/Doug-Pardee/LightZombie/issues/6#issuecomment-2844195

[Doug-Pardee]

If new Raw files are working but earlier ones aren't, I'd suspect LightZone's caching. I've had to delete funky images from the cache multiple times while trying to get the dcraw changes figured out. If I recall correctly, there's a "clear caches" button on the Preferences dialog.

[krisw]

You are correct. In the Preferences window : General : BrowserCache : Local or Global with a CLEAR button on the same line.

[krisw]

Sorry, pressed the wrong button.

[docjok]

Thanks DP, I'll try that. I was thinking that the original PEF file metadata may have been "adjusted" by LZ so that it was linked to the JPG it creates. I tried reverting back to the original file using Time Machine, but to no available. A cached file make sense; I did not realise LZ would/could do this.

krisw, I understand your comments about collaborating. I am willing to help, too, but time may be an issue. I guess if we all help out as much as as we can, we'll keep the project ticking along.

Best regards.