Xtrans support

[Beep6581]

Originally reported on Google Code with ID 2415

Today I started work at Xtrans support. This first patch only eliminates an endless
loop when trying to open a Xtrans file and a crash when trying to fast demosaic. I
set demosaic to 'nodemosaic' for Xtrans files until I get the xtrans_interpolate from
dcraw working in RT.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-08 16:31:46

---

• _Attachment: xtrans_00.patch_

[Beep6581]

Ingo I hope you can make something out of it. All I can say is that their implementation
works apparently as well as dcraw's, and is quite fast. Unfortunately it fails to export
the output image, and when I try to build a Debug version it fails to compile...

Reported by sguyader on 2014-06-19 18:30:00

[Beep6581]

I'm sure I can get the same result as dcraw's :-) Perhaps it needs some time, but I
won't give up ;-)

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-19 19:10:10

[Beep6581]

I'm sure you will, maybe even better than dcraw ;-)... I'll be patient and help you
if I can be of any help.

Reported by sguyader on 2014-06-19 19:29:10

[Beep6581]

#49 Hi squyader,

I inspected some X-T1 raw samples from Imaging Resource (all low light shots)
Looks like there are triple clipping points one at around 16380, one at around 16320
and one more (in blue channel mainly ..) at around 16210
So a safe WL should be < 16200 :) .. I would set it at 16100 to be conservative and
safe .. The highlights lost will be a negligible 0.0X stops :)

#48 Ingo hi, how did you found the matrix and the WL ?.

I would prefer to use the "official" adobe's d65 (at #23) as hardcoded matrix and use
a camconst.json item for easy changes :).
I can make it late today and also build a dcp profile ..

I just managed to complile xtrams_09.patch while been at home but now I am away again
with no compliling setup and possibly no rights to install one .. :( 

I think that reading the black level from exif instead of a hardcoded one is a priority
as is varies a lot from shot to shot(lowest 1022 - highest 1028 at the inspectred samples
.. maybe can deviate more ..)
I can see with exiftool that there are 36 BL values in RAF data (0xf00a BlackLevel
).. should be for eatch pixel of the 6X6 repeating tiles. All 36 values are equal at
the samples I inspected.
Maybe the reason that RT cannot recognize the x-trans BL is Torgers patch for Fuji
per channel BL which works fine on bayer but not on x-trans ..

I wish I could read the code  .. :) 

In my built when I change to demosaic none it still keeps to use demosaic x-trans ..
I would like to use none as a usefull and robust color rendering should be using 3X3
tiles at 33% view (with a change in green factor from 100 to 33 in channel mixer ..)

Reported by iliasgiarimis on 2014-06-20 11:45:21

[Beep6581]

Ilias,
I agree that sometimes the white level is < 16300 in the blue channel, so indeed setting
it to 16200 or 16100 should be safe.
I also agree that the black level varies a lot according to sample images, I've checked
a few images including some scenes I took with my X-T1, and sometimes de BL is around
900 (I think I remember it was even lower than 900 a couple times). So if the black
level can be read from exif instead of being fixed, it would be nice.

Reported by sguyader on 2014-06-20 12:13:36

[Beep6581]

Ilias, re #57 re #48: These are the values from X100S. I tried them for X-T1 to see,
whether the artifacts are caused by wrong WL and matrix. The final values I leave to
you :-)

I'll set demosaic method 'none' to 'none' for xtrans files in next patch, but we'll
have to make a better solution nevertheless (reflect the method in GUI etc). Actually
I used these shortcuts to allow testing before this work is done.

I'll also have a look, why the BL isn't recognized for x-trans.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-20 12:38:02

[Beep6581]

I found a solution for black levels. Next patch will use the ones from exif (in first
step only one of them, because all 36 values are equal)

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-20 13:17:55

[Beep6581]

Ingo, 

(.. this is why it hurts me my inability to read the code :( ..)

have you adapted the pre-autoWB (pre-demosaic) to the properties of x-trans ? 
https://code.google.com/p/rawtherapee/source/detail?r=7995e08a61bff7f33d1c4f1c3d161dbc601a8e8d

Reported by iliasgiarimis on 2014-06-20 14:03:49

[Beep6581]

Regarding black levels, excuse me for my ignorance, but I have an image I took indoors
à iso 1600, which shows in Rawdigger minimum values of 937,857,882 (R,G,B) while Black
Level in Exif is 1026. Why is that? Which black level should be used then?

Reported by sguyader on 2014-06-20 15:11:08

[Beep6581]

I just made a good step forwards. But before I post a new patch, I would like to ask
you, Ilias and Sebastien, if we could stay with revision da558096e08a for next patches.
Upgrading to latest revisions from Anders would mean a lot of additional work for me,
which I would like to delay just before the commit of x-trans patch.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-20 15:25:29

[Beep6581]

Sure Ingo, I'm fine we staying with rev. da558096e08a

Reported by sguyader on 2014-06-20 16:31:16

[Beep6581]

Next one. This is much closer to the result of dcraw. The relevant change was the one
Ilias mentioned in #61 (missed that before, thanks a lot, Ilias :-) Also use BL from
exif now.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-20 17:28:12

---

• _Attachment: xtrans_10.patch_

[Beep6581]

#63 yes Ingo, it will be easier for me also :) .. if I'll manage to build the compiling
setup .. never give up with rights :) 

#62 squyader, the Black level is not the same as the minimum value you can detect in
a raw file. It is the average value you take with a black frame (a shot where no light
falls on the sensor i.e in darkness with lens cover on and fast shutter speed and closed
apperture ) and this has noise which means that the values deviate up or down from
their average. The more the noise the more the deviation .. try with a higher ISO and
you'll see more deviation in raw values.

The only way to detect per shot BL in RAFs is to read the exif data ..

Reported by iliasgiarimis on 2014-06-20 17:56:44

[Beep6581]

I have to leave now so the dcp profile and camconst.json items are postponed for tomorrow.

Ingo, if you have the time can you please describe the x-trans patch steps in a human
language ??.

Reported by iliasgiarimis on 2014-06-20 18:07:14

[Beep6581]

Ilias, I'll try to do that when I'm back home in a few hours :-)

Reported by heckflosse@i-weyrich.de on 2014-06-20 18:15:36

[Beep6581]

#66 Thanks Ilias for the explanation.
#67 I already made a dcp profile for the X-T1 based on the base iso (200) shot from
Image Resources, converting the RAF file to DNG with the DNG Converter 8.4, and creating
a profile using the passport checker squares in this image with DNG Profile Editor.
The profile is attached.

Reported by sguyader on 2014-06-20 18:21:23

---

• Attachment: [Fujifilm X-T1 perso.dcp](https://storage.googleapis.com/google-code-attachments/rawtherapee/issue-2415/comment-70/Fujifilm X-T1 perso.dcp)

[Beep6581]

#65 Yes Ingo, it's much better now!

Reported by sguyader on 2014-06-20 19:12:22

[Beep6581]

As it seems to work as expected now, I'll try to make a faster version (without change
to output) tomorrow.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-20 21:54:44

[Beep6581]

I confess it is quite slow on my machine, on the other hand I run it
through virtualbox on a 3 year old windows laptop...
Le 20 juin 2014 17:54, <rawtherapee@googlecode.com> a écrit :

Reported by sguyader on 2014-06-20 22:40:22

[Beep6581]

There are some capabilities for a speedup:

Main point is that the actual version needs

4*width*height*sizeof(float) byte

for the input buffer.

It should be possible to reduce this to 

width*height*sizeof(float) + 2*c*TS*TS*sizeof(float) byte

where c is the number of threads (usually equal to the number of cores)
and TS is the size of the tiles of xtrans_interpolate (maybe some pixels bigger, we'll
see)

That would lead to much better use of available memory bandwidth and cache.

Because my machine is limited by memory bandwidth mainly (8 cores need a large memory
bandwidth), I'll try this first, because it's hard to measure other speedups before.

At my machine and with actual patch, the 3-pass xtrans_interpolate of a 16 MP file
(X-pro1, X-T1) needs about 2000 ms.
I already have a version (without the changes mentioned above), which needs only 1800
ms.

We'll see...

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-20 23:03:14

[Beep6581]

@ sguiader, 
if your RT build is for winXP32 please upload ..

Reported by iliasgiarimis on 2014-06-21 08:49:13

[Beep6581]

@ilias, sorry but even if my machine runs Windows (7 64 bit) I build RT in a Linux virtual
machine. I may try later to setup a toolchain in windows.

Reported by sguyader on 2014-06-21 10:55:17

[Beep6581]

All previous patches always showed the embedded unmodified thumb.
This patch includes the changes for normal RT thumbnail behavior for x-trans files.
Even 'Auto Distortion Correction' can be used now. I hope I didn't forget anything.
Please clear processing profiles and cache for x-trans files before test.

Last patch had modifications to dcraw.cc to get the correct BL. I moved this to rawimagesource.cc
get_colorsCoeff, so from now on the patches work with an unmodified dcraw.cc :-)

Speedup for xtrans_interpolate follows later.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-21 11:45:12

---

• _Attachment: xtrans_11.patch_

[Beep6581]

Insert the following lines in your user camconst.json in RT's cache directory besides
the options file ..
comment/uncomment the related lines for different color matrix ..

    { // Quality B
        "make_model": [ "FUJIFILM X-T1", "FUJIFILM X-E2" ],
        "dcraw_matrix": [ 8458,-2451,-855,-4597,12447,2407,-1475,2482,6526 ], // DNG
D65
//        "dcraw_matrix": [ 9289,-3279,-632,-3539,11137,2758,-1049,1950,6544 ], //
X-RITE D55
        "ranges": { "white": 16100 }
    },

attached is the dcp profile built on http://www.imaging-resource.com/PRODS/fuji-x-t1/XT1hVFAI00200.RAF.HTM
with http://xritephoto.com/ph_product_overview.aspx?ID=1257&Action=Support&SoftwareID=986

Reported by iliasgiarimis on 2014-06-21 15:17:21

---

• _Attachment: [Fujifilm X-T1_IR-VFAI_X-RITE.dcp](https://storage.googleapis.com/google-code-attachments/rawtherapee/issue-2415/comment-79/Fujifilm X-T1_IR-VFAIX-RITE.dcp)

[Beep6581]

I would like to make a xtrans version of Flat-Field. Should be no problem, but I've
no picture to test.

http://50.87.144.65/~rt/w/index.php?title=Flat_Field

Sebastien, could you make a few shots? (but no need to hurry :-)

I need:

For vignetting correction with Flat-Field
1.) a real image shot at low iso and with smallest focal length and largest aperture
(F2.0 or whatever your largest aperture is) 
2.) a Flat-Field shot with the same settings

For dust removal with Flat-Field

1.) a real image shot at low iso and with small aperture (F11)
2.) a Flat-Field shot with the same settings

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-21 15:51:49

[Beep6581]

OK I'll make the shots, however I don't know if I'll be able to upload them before Monday,
I'll see what I can do.
The widest focal length and aperture will be 18mm f/2.8. Is that OK ?

Reported by sguyader on 2014-06-21 16:04:07

[Beep6581]

Sebastien, no need to hurry.

18mm f/2.8 should be perfect for the vignetting correction flat field.
The dust removal Flat-Field shot only makes sense, if you have dust spots on the sensor
of your cam. In this case, F11 or F16 would make sense for testing purposes.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-21 17:38:14

[Beep6581]

I put 4 RAF files in my gdrive :
https://drive.google.com/folder/d/0B_AvPFlUj8t5RC1pbHcxUkxYcGs/edit
Le 21 juin 2014 13:38, <rawtherapee@googlecode.com> a écrit :

Reported by sguyader on 2014-06-22 16:17:59

[Beep6581]

@ Ilias, today I'll try to setup a build environment on Windows and compile
a Windows binary.

On Sun, Jun 22, 2014 at 12:17 PM, Sebastien Guyader <sguyader@gmail.com>
wrote:

Reported by sguyader on 2014-06-23 11:29:48

[Beep6581]

@ sguyader, thanks

If you can build for win32 it will be usefull for me :)

Reported by iliasgiarimis on 2014-06-23 13:52:16

[Beep6581]

It's not going to be easy... Right now I can't succeed in configuration
prior to compilation... I already have a build environment which seems not
compatible with RawTherapee (for building other software)... I'll try to
uninstall and reinstall according to RawTherapee's documentation.

Reported by sguyader on 2014-06-23 14:52:47

[Beep6581]

Reported by heckflosse@i-weyrich.de on 2014-06-23 20:23:48

• Blocking: #2391

[Beep6581]

Next one. No reasonable speedup, but revised the code and added support for flat field
correction. It would also be possible to add Dark-Frame support, but I've no images
to test this, so I disabled Dark-Frame for xtrans.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-23 21:35:29

---

• _Attachment: xtrans_12.patch_

[Beep6581]

Another thing to be solved for xtrans files is the lens information. Actually it shows
'Unknown' in Info-Dialog, though 'LensModel' in Exif shows the Lens correctly. Shouldn't
be a big problem, but still open.

Reported by heckflosse@i-weyrich.de on 2014-06-23 23:09:13

[Beep6581]

#93 if you want I can take a darkgrame picture for you to test.
#94 if lens is not recognised, how does distortion autocorrection work ?
Le 23 juin 2014 19:09, <rawtherapee@googlecode.com> a écrit :

Reported by sguyader on 2014-06-24 01:40:47

[Beep6581]

Hmmm .. it could be usefull to have a series of blackframes at all ISO's and if possible
both at short and long exposures :)

may be this can help to characterize these sensors better ..  

Reported by iliasgiarimis on 2014-06-24 07:27:58

[Beep6581]

#95: a darkframe picture would be good. 
The auto Distortion correction works by comparing the embedded thumb (which is corrected
in some cases) to the thumb generated from raw data.

Reported by heckflosse@i-weyrich.de on 2014-06-24 07:46:00

[Beep6581]

Ok I'll try and take a bunch of darkframes. What shutter speed do you want me to use
for short and long exposures?

Reported by sguyader on 2014-06-24 11:44:28

[Beep6581]

Sebastien, I don't know at which exposure time your camera produces thermal noise, but
I guess a real life shot and a corresponding dark frame shot both shot with 15 seconds
exposure time and lowest ISO should do it.

Reported by heckflosse@i-weyrich.de on 2014-06-24 12:02:24

[Beep6581]

I'm uploading several shots taken at 15s-f/22.0, isos 200, 400, 800, 1600, 3200 and
6400, with lens cap off ("Light") and on ("Dark"). The raf files are here: https://drive.google.com/folderview?id=0B_AvPFlUj8t5eWotclNVVG1mMWM&usp=sharing
Let me know if that's good.
You'll notice a hot/stuck pixel around coordinate (2663,1461), the good news is that
the hot/dead pixel filter in Preprocessing works and removes it correctly!

Reported by sguyader on 2014-06-24 15:48:29

[Beep6581]

Sebastien, the light shots are too light, but no problem, I tested the Dark frame on
the Dark frame picture itself. Gives a nice black picture :-) Will post a patch in
the evening.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-24 16:32:53

[Beep6581]

This patch includes the following changes:

1.) Support for Dark Frame subtraction
2.) Avoid clipping when flat field is applied to a bright (but not overexposed) image
(actually only for xtrans, but I'll make this for bayer sensors too)
    (Sebastien, you can evaluate this change by applying DSCF0946.RAF as flat field
to DSCF0947.RAF before and after this patch. Before patch has a clipped region in lower
right corner, after patch has not.
3.) Flat field correction had a bug for images with different black levels. That's
not the case for xtrans files, but I saw it and fixed it with this patch

There's still an Issue with Flat Field Blur Type 'Horizontal + Vertical' (color cast),
but I guess that's independent on xtrans and only occurs when black levels are not
zero (which is the case for xtrans files, but also for the files of some other cams)..

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-24 21:38:29

---

• _Attachment: xtrans_13.patch_

[Beep6581]

#102: Both Dark Frame Substraction and Flat Field correction work. Indeed before patch
13 the Falt Field correction was clipping regions, since patch 13 it works, though
I can reproduce the color cast with Horizontal + Vertical blur type.
Overall, I've seen a speed increase since patch 11 or 12 (I don't remember which) both
in fast demosaicing for preview (it's really fast now) and also full demosaicing at
full size.
Great job Ingo!

#75: Ilias I'm trying to build a Win32 binary, but it's driving me nuts to get the
dependencies (fftw3f, lcms2 and iptcdata libraries are not detected despite being present
in the correct path)

Reported by sguyader on 2014-06-25 11:16:11

[Beep6581]

This time only changes to xtrans_interpolate. About 10% faster than last version. Memory
reduction during xtrans_interpolate reduced by 2*width*height*sizeof(float) byte.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-25 13:13:42

---

• _Attachment: xtrans_14.patch_

[Beep6581]

Ilias, I finally succeeded and compiled a 32-bit executable for Windows. It includes
the latest patch from comment #104, your dcp profile, and your additions to camconst.json.
I works flawlessly on my laptop (Windows 7 64 bit). You can download it from here:
https://drive.google.com/file/d/0B_AvPFlUj8t5ZGxURU1Fdy1mOEE/edit?usp=sharing

@Ingo, is it possible to share this binary with more people, or should I wait untill
a more official release? I know people on depreview forums are waiting for this.

Reported by sguyader on 2014-06-25 15:00:51

[Beep6581]

I don't know if the issue is related with x-trans, but currently when Chromatic Aberration
correction doesn't work (neither manual nor automatic). Also, when I use False Color
Suppression, it does introduce more chromatic aberration where there is some. If you
want to try, take the dpreview iso 200 raf file (https://drive.google.com/file/d/0B_AvPFlUj8t5a1Y4N2g1RzNCWFk/edit?usp=sharing)
and look at the headshot of the black guy at the left of the image. There's some trace
of red/blue aberration along the right and left edges of the photograph, these are
not corrected by the correction tool, and get reinforced when applying False Color
suppression

Reported by sguyader on 2014-06-25 18:30:19

[Beep6581]

Sebastien, Chromatic Aberration Correction is actually disabled for x-trans. I try to
make a x-trans version for Chromatic Aberration correction, but I can't say, whether
I will be successful. 
About the False Color Suppression: It's not made for CA-correction. I've tested with
the raf you mentioned and it spreads the CA a bit, so it less pronounced, but a bit
wider.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-25 19:14:27

[Beep6581]

@ squyader, thanks

RT dll's are missing from your link :(

I tried copying the dlls from my previous compile but I don't have bzip2.dll and RT
cannot start. 

Reported by iliasgiarimis on 2014-06-25 19:34:19

[Beep6581]

@ Ilias, sorry I forgot about the dlls.

I downloaded:
- RT dependencies from http://www.rawtherapee.com/shared/builds/windows/dependencies_for_creating_builds/RT-dependencies-WinXP32-i686-gtkmm2.22.7z
- Gtkmm-devel http://ftp.gnome.org/pub/GNOME/binaries/win32/gtkmm/2.22/gtkmm-win32-devel-2.22.0-2.exe
- bzip2 from http://en.sourceforge.jp/projects/sfnet_gnuwin32/downloads/bzip2/1.0.5/bzip2-1.0.5-setup.exe/

I hope you'll be fine with that.

Reported by sguyader on 2014-06-25 20:05:44

[Beep6581]

Now I can start RT but it crashed :(

Reported by iliasgiarimis on 2014-06-25 20:23:34