Add camconst support for foveon files

[Beep6581]

Originally reported on Google Code with ID 2729

This patch adds camconst support for foveon files.
Ilias, could you add the camconst entries for the foveon cams please?

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-30 17:13:41

---

• _Attachment: camconst_foveon.patch_

[Beep6581]

Last patch had a bug in thumbnail generation when a raw crop was defined in camconst.json
for a foveon cam. Fixed.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-30 19:04:38

---

• _Attachment: issue2729_01.patch_

[Beep6581]

Ingo can you confirm if dcraw.cc reads/uses the color matrix from the exif data when
we do not define one in camconst.json ?.

I see abnormal temp/tint and this is usually an indication of a wrong (or not existing)
color matrix.

Reported by iliasgiarimis on 2015-03-30 20:57:04

[Beep6581]

Ilias, afaik it uses this one: 

dcraw.cc line 7724

  /* index 0 -- all Foveon cameras */
  { 1.4032,-0.2231,-0.1016,-0.5263,1.4816,0.017,-0.0112,0.0183,0.9113 },

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-30 21:02:52

[Beep6581]

Ilias, confirmed #3 by looking at verbose output:

rgb_cam:[ [ 1.403200 -0.526300 -0.011200], [-0.223100 1.481600 0.018300], [-0.101600
0.017000 0.911300] ] (not bayer)

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-30 21:38:55

[Beep6581]

Ilias, I had a look at the dcraw foveon code. It seems we have to call the foveon_interpolate
function too. But there the problems begin. It doesn't find "WhiteBalanceIlluminants"
for SD1 Merrill and then stops the function, leaving the image unchanged. I found this:
http://www.dpreview.com/forums/post/41138764
Perhaps I take some time to dig into that stuff...

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-31 17:59:54

[Beep6581]

Hi Ingo, if we want RT to become a reliable foveon converter there is much to do ..

May be digging in related code http://www.proxel.se/x3f.html will help ..  it's open
source but under "BSD lookalike license"

For a properly decoded foveon data some cryptic exif info must be parsed and decoded.
Ronald Karlsson (the author of x3f tools) suggested to read the side optically black
data to calculate the BlackLevel http://www.dpreview.com/forums/post/55485828
although I cannot find the proper area where we can read the correct value .. 

For the moment I stole the correct BL=17 and built a better icm profile adapted to
this BL .. 
https://drive.google.com/file/d/0B0NqktTgc54sa0JzWlVYVjN0QTA/view?usp=sharing

Some black frames from foveon users will help ..

Is the "save reference image for profiling" aborting the color matrix at #3 ??. Or
is it used and then my profile is a correction profile upon the matrix ?.

I also saw Dcraw aborting a part of the code .. as I understand by the comments it
is about color interpolation .. so I just found a workaround by using RT's .. color
denoise. It works acceptably :) 

Reported by iliasgiarimis on 2015-03-31 20:08:34

[Beep6581]

Hi Ilias, I think we already made a good first step now. As you say there is much to
do, but it hasn't to be done immediately. With your new icm profile it already looks
'good' for SD1 Merrill. Perhaps users in forum can supply black frames too.

About 'reference image for profiling' I have to look at the code. Hopefully I can answer
your question tomorrow.

Shall I commit the foveon camconst patch or do you want to add some black levels and
crops to camconst.json?

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-31 20:37:49

[Beep6581]

Ingo,

Wait some hours for camconst.json .. tonight I will add black levels and raw crops
for as many foveon models I can find samples of.

Now the only serious defect is the WB which falls out of range .. maybe we should try
with inverting the matrix #3 ?? ..
I had the hope that copying the matrix from the icm profiles in camconst.json (see
last comments by Anders .. https://code.google.com/p/rawtherapee/source/detail?r=0bdc98a3d57ee55dc0ea044ab73a2fecd1257dd4
)
would do the job but failed .. maybe an matrix inversion is needed ?. 

Reported by iliasgiarimis on 2015-03-31 20:52:00

[Beep6581]

Ilias, no hurry to commit. Take your time to make the changes.
I'll have a look at the matrix now.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-31 21:09:06

[Beep6581]

Ilias, I tried the inverted matrix in dcraw code. The auto wb value changed from about
49000 to 24000, but when using auto wb the output was identical, independent on matrix!
When using fixed wb, the output was different of course.
So I guess, if we can build a correct matrix, the auto wb value will also be in normal
range.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-03-31 22:09:41

[Beep6581]

OK I lost my time with this alien WB :( .. But I came to a dirty solution .. :-)

The problem is that the WB algo is not tuned for layered sensors where the possibility
of a photon to get registered at the deepest level (red color) decreases exponentially.
So for a neutral grey surface under daylight we take R=0.5, G=1.0, B=2.0 instead of
a usual bayer which gives R=2.5 G=1.0 B=1.5. This is a extreme 5X difference for R
.. These abnormal multipliers are the reason that the WB algo get crazy ..

The solution is to insert a positive bias for R and G to adapt the values to the WB
algorithm's operative range. This is exactly what we can do for R with the "red/blue
equalizer" slider when we push to the right. But still the bias we need for G is missing.

The dirty solution I came with is to multiply by 1.5 the G and by 1.5*1.5=2.25 the
R when calculating a matrix from the icm profile at #6
( mutiply the icm's matrix with [ 2.25, 1.5, 1.0 ] then bradford adapt D50 to D65,
then invert ..). Use the result in camconst.json. Of course this matrix is only usable
for WB calculations while the color rendering it gives is terrible. But we have the
icm profile for correct colors :)

Please try ..

    { // Quality C, frame corrections, color matrix only usable for WB calculations
..
        "make_model": "Sigma SD1 Merrill",
        "dcraw_matrix": [ 15440,-7911,-3450,3593,3228,-512,-3726,9191,5084 ], // experimental,
WB-preconditioned, inverted icm's matrix
      //  "dcraw_matrix": [ 14032,-2231,-1016,-5263,14816,170,-112,183,9113 ], // 
hardcoded internal
        "ranges": { "black": 17, "white": 4070 },
        "raw_crop": [ 12, 52, -110, -8 ]
    },  

Reported by iliasgiarimis on 2015-04-01 06:25:50

[Beep6581]

Camconst.json for some merrills ..
The black level (at the samples I inspected) is either 16 or 31 but maybe more depending
on settings, temperature etc. RT should measure it per frame at the horizontal optically
black stripe at the bottom or top (depending on model, if both top and bottom exist,
use the largest. 
We will need to add "masked_areas": [ x, x, x, x ] when enabled.

    { // Quality c, correction for frame width
        "make_model": "Sigma SD1",
        "dcraw_matrix": [ 15440,-7911,-3450,3593,3228,-512,-3726,9191,5084 ], // experimental
inverted icc
      //  "dcraw_matrix": [ 10448,-7484,10162,-5403,25269,-38326,5487,-6956,37375 ],
//  icc 1.04477,-0.74838,1.01617, -0.54028,2.52690,-3.83257, 0.54869,-0.69556,3.73746
      //  "dcraw_matrix": [ 14032,-2231,-1016,-5263,14816,170,-112,183,9113 ], // 
hardcoded 
        "ranges": { "black": 16, "white": 4070 },// BL is 16 or 31, should be measured
at the horizontal black stripe at the top 
        "raw_crop": [ 12, 52, -110, -8 ]
    },
    { // Quality C, correction for frame width, color matrix missing ..
        "make_model": "Sigma SD1 Merrill",
        "dcraw_matrix": [ 15440,-7911,-3450,3593,3228,-512,-3726,9191,5084 ], // experimental
inverted icc
      //  "dcraw_matrix": [ 14032,-2231,-1016,-5263,14816,170,-112,183,9113 ], // 
hardcoded 
        "ranges": { "black": 16, "white": 4070 }, // BL is 16 or 31, should be measured
at the horizontal black stripe at the top 
        "raw_crop": [ 12, 52, -110, -8 ] // for small size all numbers/2 
    },
    { // Quality C, correction for frame width, color matrix missing ..
        "make_model": [ "Sigma DP1 Merrill", "Sigma DP2 Merrill", "Sigma DP2 Merrill"
],
        "dcraw_matrix": [ 15440,-7911,-3450,3593,3228,-512,-3726,9191,5084 ], // experimental
inverted icc
      //  "dcraw_matrix": [ 14032,-2231,-1016,-5263,14816,170,-112,183,9113 ], // 
hardcoded 
        "ranges": { "black": 16, "white": 4070 }, // BL is 16 or 31, should be measured
at the horizontal black stripe at the bottom 
        "raw_crop": [ 12, 0, -110, -62 ] // for small size all numbers/2 
    },

Reported by iliasgiarimis on 2015-04-01 10:07:03

[Beep6581]

Question which interests me - what is method of white balancing of Foveon images?

I know that dual illuminant is suitable for most Bayer sensors, I know that even just
channel scaling is suitable for Bayer sensors in most cases.

But good adaptation of Foveon output to RGB space is not possible with use of matrix
conversion, Foveon output needs LUT profile instead. And now comes my question:
1) what is white balancing method for Foeon output?
2) if it is simply channel scaling, does it affect profiling quality?

Reported by pinhuer on 2015-04-01 13:22:56

[Beep6581]

3) am I right in that Foveon output requires LUT profile and not just matrix?

Reported by pinhuer on 2015-04-01 13:23:38

[Beep6581]

http://www.cvc.uab.es/color_calibration/SpaceConv.htm - this page enlightens me about
matrix conversions. I of course mean 3x3 matrices.

Reported by pinhuer on 2015-04-01 14:16:04

[Beep6581]

Some changes:

1) Foveon thumbnail was wrong when auto wb was used
2) Disabled CA-Correction for Foveon
3) Set border for Foveon to 0 instead of 4 (Bayer)
4) included the new camconst.json entries
5) some additional corrections and code cleanups

Ingo

Reported by heckflosse@i-weyrich.de on 2015-04-01 20:22:36

---

• _Attachment: issue2729_02.patch_

[Beep6581]

Ingo, is it easy/possible to enable (for foveon) the black levels in raw tab ?. As a
first workaround for the differing black levels (16 vs 31) ..

Are the flat_field/black_frame tools enabled for foveon ?.

Reported by iliasgiarimis on 2015-04-02 09:47:39

[Beep6581]

Ilias. Yes, it's easy. Yes, ff and df are enabled for foveon, but not tested.
I'll make a new patch with black levels sliders

Reported by heckflosse@i-weyrich.de on 2015-04-02 09:53:56

[Beep6581]

Ilias, here's the patch. I activated the black level sliders in raw/bayer tab for foveon.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-04-02 10:02:54

---

• _Attachment: issue2729_03.patch_

[Beep6581]

Dark frame seems to work, but Flat field does not. I'll have a look.

Reported by heckflosse@i-weyrich.de on 2015-04-02 10:22:39

[Beep6581]

Addition: Dark Frame single file works, but Dark Frame auto selection does not

Reported by heckflosse@i-weyrich.de on 2015-04-02 10:47:07

[Beep6581]

please respond

Reported by pinhuer on 2015-04-02 13:22:53

[Beep6581]

Pinhuer, WB is simply channel scaling. About the LUT and matrix, Ilias knows better
than I do.

Reported by heckflosse@i-weyrich.de on 2015-04-02 13:29:22

[Beep6581]

Ilias, experimented by building both matrix + gamma vs lab CLUT and found the first
better :).
I have no idea if there is a theoretical advantage for labCLUT ..

All icms are available ..
https://drive.google.com/folderview?id=0B0NqktTgc54sUmtfZlZjLUhESzQ&usp=sharing

I used RT's reference tiff, then opened in PS4 with no color management, cropped, rotated,
increased exposure so that the mid grey patch is at L=18 and used curves to WBbalance
at the same patch (mid tones picker ..). Then coca for agryllcmsGUI .. 

Reported by iliasgiarimis on 2015-04-02 15:25:03

[Beep6581]

Heckflo, OK.

Ilias, you may be interested in that Photoshop's exposure is not exposure in terms
of RAWTherapee but multiplication of gamma-corrected data instead. I am not following
the discussion but it may be related to what you do.

Reported by pinhuer on 2015-04-02 16:02:51

[Beep6581]

Hi pinhuer,

The gamma of the reference shot is 1.0 so it should be totally linear .. no ?

Reported by iliasgiarimis on 2015-04-03 02:26:28

[Beep6581]

Photoshop's exposure is correct only if the Gamma of profile is 1. If Gamma is 1 the
all curves are perfectly linear too.

But if you know that or if that does not affect what you do - do not mind what I said.

Reported by pinhuer on 2015-04-04 13:35:21

[Beep6581]

Don't you know that "save reference image for profiling" exports in gamma = 1.0 ?? !!.

Reported by iliasgiarimis on 2015-04-04 16:39:16

[Beep6581]

I don't, you made me to check it.

I found that RT does not append any profile to the image which is written upon mentioned
button press. It means that Photoshop uses it's "working" profile as image profile.
If you did not change it it is sRGB with default gamma. Which is not 1.
______

All this means that
Photoshop's exposure and curves are accurate and
grey values are correct too and
image exported correctly but
the result is not displayed correctly in Photoshop.
______

I feel like you know all of this already.

Reported by pinhuer on 2015-04-04 16:49:19

[Beep6581]

Just to add my two cents:

Please check my samples at http://rawtherapee.com/forum/viewtopic.php?f=1&t=5958&p=41515#p41515

The current white balance for the SD1M (Merrill sensor) is quite off...

With the correct white balance and three ICC profiles for tungsten/sunny/cloudy I think
we have 99% of cases covered.

Maybe you want also a sample of shade/blue-sky illuminant? But that changes a lot...

Reported by rnbc.r0 on 2015-04-06 00:06:08

[Beep6581]

Digging in samples is starting ..

although a critical part will be flat fields to correct foveon's periphery casts which
samples are missing ..
BTW is the flat field tool enabled for foveon ?

Ingo, is it easy to 
a) enable for foveon the "masked_areas" (same as with Canon cameras) Sorry if it's
already enabled .. I haven't checked yet .. 
b) push Dcraw.cc when it cannot gather the WB from exif, to use default daylight WB
multipliers (i.e. R=3.4 G=1.9 B=1.0) and continue with the now abandoned process ?.

I think that possibly the hardcoded matrix (#3) is an RGB matrix and then the WB multipliers
are needed to convert to Dcraw_matrix (XYZ)

Meanwhile, I need help to calculate "dcraw_matrix" from the matrix in icms that I build
.. Hopefully Jacques or Anders will read it ..   

Reported by iliasgiarimis on 2015-04-06 20:43:01

[Beep6581]

According to my samples the green multiplier is not correct. The correct multipliers
for sunlight are 3.40,1.71,1.0 (or 1.0, 0.503, 0.294).

It's difficult to get proper flat fields for all situations because it depends on the
exit pupil virtual distance and aperture. For telescopic lenses this isn't even an
issue!

And all my lenses are rather old. Only one is currently in production, I think, the
10-20 f/4.0-5.6 is still sold. The others are long discontinued.

30mm f/1.4, there is a new model now, with different optics.
180mm f/3.5 macro, long discontinued.
10-20 f/4.0-5.6, still sold, but there are other newer (and better?) designs in Sigma's
lineup.
18-50 f/2.8 (non OS), long discontinued.
24-70 f/2.8, my model isn't sold since 2006 or so... since the film days.

So any samples I could get you are probably useless :P

Flat field partially solves the issue (if it works?) and can be made by the camera
owner.

Reported by rnbc.r0 on 2015-04-06 22:26:44

[Beep6581]

As I wrote above, actually Flat fields for foveon don't work. But it's on my todo list.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-04-06 23:01:51

[Beep6581]

Hello, I am new to this forum although I have talked to Ilias at DPR about Merrills.

Since then I have sold my SD1 Merrill but kept my early SD9 and SD10 which use X3F
format FOVb. I tried Ilias' download 4.2.174 but it crashes when opening just after
the version warning comes up (Windows XP Pro, 32-bit, Intel Pentium 4, 4GB RAM, extended
addressing enabled, I think).

My question, a little off-topic, is there a development version, other than the one
I tried, which will run on my system and can see and open my files (not interested
in Merrills).

Or is there something I can added to make the version above work? I am not a programmer
and only semi-computer literate :-(

Thanks,

Ted

Reported by xpatUSA on 2015-04-07 17:33:40

[Beep6581]

Um, sorry, I think it was 4.2.143 that crashed.

Getting too old for this sruff ;-)

Reported by xpatUSA on 2015-04-07 17:39:44

[Beep6581]

xpatUSA, if RT crashes on start, please have a look at http://rawpedia.rawtherapee.com/How_to_fix_crashes_on_startup.
If the crash is related to Foveon files, with 4.2.144 I fixed a bug, which could cause
undefinef behaviour.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-04-07 20:50:24

[Beep6581]

Hello, just to add a few more info regarding old x3f files support.

As I told here:

http://rawtherapee.com/forum/viewtopic.php?f=1&t=5977

There is an issue with SD14 files which makes the pixels with values near zero (dark)
appear as brightly colored. I suspect (because I once had the same issue in my conversion
pipeling implemented using ImageMagick/HDRI_version) this is a generic problem in the
conversion of negative float values into integers. In "convert" this was easily solved
using "-clamp", which clamps negative values to zero.

You can check an example x-rite target photographed with a SD14 under sunlight here:

http://rnbc.dynip.sapo.pt/pub/x3f-examples-02/

The pixels in the borders are masked, and have values near zero, bur appear as brightly
colored. Also some other dark pixels in the image appear brightly colored.

Reported by rnbc.r0 on 2015-04-07 22:31:03

[Beep6581]

rnbc.r0, I've problems downloading the files you uploaded to http://rnbc.dynip.sapo.pt/
They download but don't appear in rt file browser. Can you upload them again to filebin.net
or another free service?

Ingo

Reported by heckflosse@i-weyrich.de on 2015-04-07 22:44:45

[Beep6581]

Hello Ingo!

Err... well, that's a computer at my home, running apache on debian. I've added a tar
with both files, just in case your browser is doing tricks on your back. Their MD5
checksums are also there, MD5SUMS.txt

I have confirmed the checksums in my windows laptop, where I run rawtherapee under
windows / cygwin, and they are the same.

Can you try again please? And check the MD5SUM of the files after downloading.

Reported by rnbc.r0 on 2015-04-08 00:00:34

[Beep6581]

Ingo said "xpatUSA, if RT crashes on start, please have a look at http://rawpedia.rawtherapee.com/How_to_fix_crashes_on_startup.
If the crash is related to Foveon files, with 4.2.144 I fixed a bug, which could cause
undefinef behaviour."

Thanks Ingo,

Forgot to mention that I've been running 4.2.3 with problems.

But, I tried the "make an empty folder C:\\foo" test and 4.2.143 opened up OK with
foo as the selected folder. But as soon as I tried to open foo with any file in it,
RT crashed immediately. Or any other file with files that 4.2.3 sees and opens, same
thing.

Since I am at the limit of my knowledge in this kind of thing I am not asking for more
assistance right now and will go back to 4.2.3.

Very interesting and thank you for your help!

Ted

Reported by xpatUSA on 2015-04-08 01:39:59

[Beep6581]

Camconst.json item for SD14 .. (Rodrigo's files worked fine at my system)

    { // Quality c, corrections for frame size, black level not declared properly we
have to try with BL=37 also
        "make_model": "Sigma SD14",
        "dcraw_matrix": [ 16411,-4764,-2383,8110,2603,-645,3135,3878,1984 ], // experimental
inverted icc wp12 - build with BL=15
      //  "dcraw_matrix": [ 13804,-4156,-1896,6917,1909,-431,2768,2989,1741 ], // experimental
inverted icc wp10 - build with BL=15
        "ranges": { "black": 15, "white": 4080 },// BL is 16 or 31, peripheral black
stripes give BL around 37 
        "raw_crop": [ 18, 12, 2652, 1768 ]
    },

Find attached the icm profiles .. I think the wp12 is better (keeps tonality better)

@ Ted
win32 build with the 03 patch + the above camconst entry for SD14
https://drive.google.com/file/d/0B0NqktTgc54sRlVrXzh1LUNBQ1k/view?usp=sharing
I hope this one works, maybe it was the icc directory which I delete in my setup because
it consumes some more memory. This change works on win vista but it happened again
to crash on winXP

I have limited time now to continue with Merrill details and new profiles and I will
be on a trip for a week .. 

Reported by iliasgiarimis on 2015-04-08 03:21:49

---

• Attachment: sd14-bl15-crop-matrix-gamma-wp12.icm
• Attachment: sd14-bl15-crop-matrix-gamma-wp10.icm

[Beep6581]

Updated camconst items for merrills and sd1-sd14, includes color matrices calculated
from the XYZ data of icms which gives "correct" rendering and WB ..

   { // Quality c, corrections for frame size, black level not declared properly
        "make_model": "Sigma SD14",
        "dcraw_matrix": [ 16411,-4764,-2383,8110,2603,-645,3135,3878,1984 ], // experimental
inverted icc wp12 - build with BL=15
      //  "dcraw_matrix": [ 13804,-4156,-1896,6917,1909,-431,2768,2989,1741 ], // experimental
inverted icc wp10 - build with BL=15
        "ranges": { "black": 15, "white": 4080 },// peripheral black stripes give BL
around 37 
        "raw_crop": [ 18, 12, 2652, 1768 ]
    },

    { // Quality c, correction for frame width
        "make_model": "Sigma SD1",
        "dcraw_matrix": [ 5270,42,-814,3737,5506,124,1112,9714,4510 ], // experimental
from icm 1.04477,-0.74838,1.01617, -0.54028,2.52690,-3.83257, 0.54869,-0.69556,3.73746
        "ranges": { "black": 16, "white": 4070 },// BL is 16 or 31, should be measured
at the horizontal black stripe at the top 
        "raw_crop": [ 12, 52, -110, -8 ]
    },
    { // Quality C, correction for frame width, color matrix investigated ..
        "make_model": "Sigma SD1 Merrill",
        "dcraw_matrix": [ 7211,-1577,-769,4996,3428,440,2717,7117,4699 ], // experimental
inverted icc cloudy8140 d65
      //  "dcraw_matrix": [ 5666,139,-892,3780,5428,270,1366,9757,4526 ], // experimental
inverted icc sunny8161
      // "dcraw_matrix": [ 10288,-2449,-1718,8976,1868,-1608,7011,5039,-249 ], // experimental
inverted icc tungsten8130 wp11
      // "dcraw_matrix": [ 5864,679,-1491,2963,7635,-919,-640,13738,2790 ], // experimental
inverted icc sunny8160
      //  "dcraw_matrix": [ 14032,-2231,-1016,-5263,14816,170,-112,183,9113 ], // 
hardcoded 
        "ranges": { "black": 16, "white": 4070 }, // BL is 16 or 31, should be measured
at the horizontal black stripe at the top 
        "raw_crop": [ 12, 52, -110, -8 ] // for small size all numbers/2 
    },
    { // Quality C, correction for frame width, color matrix guessed ..
        "make_model": [ "Sigma DP1 Merrill", "Sigma DP2 Merrill", "Sigma DP3 Merrill"
],
        "dcraw_matrix": [ 7211,-1577,-769,4996,3428,440,2717,7117,4699 ], // copy fron
SD1 Merrill icc cloudy8140 d65
        "ranges": { "black": 16, "white": 4070 }, // BL is 16 or 31, should be measured
at the horizontal black stripe at the bottom 
        "raw_crop": [ 12, 0, -110, -62 ] // for small size all numbers/2 
    },

 SD1_Merrill CC24 crops, Sunny-Cloudy-Tungsten Icm profiles plus calculation spreadsheets
..
https://drive.google.com/folderview?id=0B0NqktTgc54sfjNLU0dwS3J0eXlsLURQYjBvaVBRbmtEWjBxV0NocldSOXhjb3dIZXYtN0U&usp=sharing

Reported by iliasgiarimis on 2015-04-08 08:10:34

[Beep6581]

Just compiled the latest version from Mercurial in Ubuntu 14.04 LTS and grabbed the
additions to camconst.json

Problems:

The line for SD14 in camconst.json doesn't seem to be used by the program at all...
perhaps the model isn't recognized?

IIRC the black point for SD14 is zero, not 16 or 31. Also I'm not sure about the white
point. I'll get a overexposed sample and post it today for you to better evaluate the
white point.

I still get the same problem in the near-black pixels of SD14: they get brightly colored.
Otherwise the colors are fine, especially with a ICC file :)

The near-black-pixels problem occurs, for example, with the sample at http://rnbc.dynip.sapo.pt/pub/x3f-examples-02/,
both in the borders (masked pixels) and in some darker areas. But it happens also with
all other files I tested at home...

Reported by rnbc.r0 on 2015-04-08 12:51:39

[Beep6581]

re #40: Download of zip worked fine :-)

Here's a new patch, which includes latest camconst.json entries posted by Ilias in
#43
and fixes the bug from #38 (near black pixels).

Ingo

Reported by heckflosse@i-weyrich.de on 2015-04-08 13:56:45

---

• _Attachment: issue2729_04.patch_

[Beep6581]

re #45: At my system the lines for SD14 in camconst.json get used

Reported by heckflosse@i-weyrich.de on 2015-04-08 14:18:21

[Beep6581]

Ok, tested your patch.

This is the current status for SD14.

Things that are ok:

-- camconst.json works for SD14.
-- Dark pixels on SD14 fixed.
-- Current matrix has good colors.

Things that are not ok:

-- Black point for SD14 is 0, ZERO, not 15! Just tested with 0 and it is the correct
value. This is just a simple change in camconst.json, no problem. Just change it :)
-- No exif data...

Things I don't know:

-- Need to test white point for the sensor. I will do it soon.

Reported by rnbc.r0 on 2015-04-08 15:16:25

[Beep6581]

Issue 1672 has been merged into this issue.

Reported by heckflosse@i-weyrich.de on 2015-04-08 18:07:58

[Beep6581]

re #49: Let's try to make a complete todo list for foveon:

-- exif data
-- Flat Field support
-- hot/bad pixels support
-- calculate black point (from masked masked borders) if possible, instead defining
it in camconst.json

To be continued...

Ingo

Reported by heckflosse@i-weyrich.de on 2015-04-08 18:28:07

[Beep6581]

Seems correct to me!

Exif data: dcraw already dumps that data, I think? Otherwise it's available in the
embedded JPEG: perhaps it's easier to extract from there for you?

Flat field should have 3 channels, of course, since the sensor has 3 different layers.
The same for black frames.

Hot/Bad pixels can be detected finding pixels with very high saturation, which never
exist naturally in these sensors, because the color filters are broad band. But this
is just a hint. I'm sure you have better ideas!

The black point can be derived from the masked borders, of course, truncating at 3-sigma
below the average, perhaps? One should not truncate at the average, because there is
valuable image data below the noise threshold, and that can give shadows a more "organic"
look I think.

Specifically for the SD14 the camera already truncates a bit too early, IMHO, dumping
valuable data... so you can effectively set it at zero. But that's something users
can override in the Raw Black Points control, right?

Best regards,

Rodrigo

Reported by rnbc.r0 on 2015-04-09 00:02:28

[Beep6581]

I have now included a SD14 sample to determine white point, which is about 6700, not
4070!

http://rnbc.dynip.sapo.pt/pub/x3f-examples-02/

Please also check this old thread in Dpreview:

http://www.dpreview.com/forums/thread/3049750

For the SD14 I propose:

"ranges": { "black": 0, "white": 6700 },

Reported by rnbc.r0 on 2015-04-09 00:31:39