Corrupted TIFF when converting some FUJI Raws

[Beep6581]

Originally reported on Google Code with ID 1035

Discussion see here: http://rawtherapee.com/forum/viewtopic.php?p=24435

RT does not write that interop tiff tag ($A005, it is optional) itself, but copies
it over if initial file has it.
That causes problems when converting some FUJI raw files.
This patch disabled the copying of this tag if existing.

There are some preexisting TODO remarks in RTs TIFF code, so the TIFF code will need
some more overhauls. But that’s something for a RT5 rewrite I guess.

Reported by oduis@hotmail.com on 2011-10-08 07:43:31

---

• Attachment: TiffFuji.patch

[Beep6581]

The hosting website wouldn't let me download the Fuji_0032.RAF file but the patch itself
applied and compiled fine in Gentoo 64.

--update 
Found some raws here: http://www.rawsamples.ch/html/de/fuji.html
Downloaded a few but none of them showed anything when I ran:
exiftool -H *.raf | grep 005

How do you get exiftool to show just tag 0xa005?

Reported by entertheyoni on 2011-10-08 22:56:51

[Beep6581]

Hi Slony,

Thanks for testing. The interop 0xa005 is a parent tag that has child tags. Exiftool
like that will only show the child ids 0x0001 and 0x0002 of the RAW:
0x0001  Interoperability Index  R98 - DCF basic file (sRGB)
0x0002  Interoperability Version    0100

The sample provided was made with a Finepix S9600.
If you just fire up exiftool on the output TIFF it would have shown this warning in
the output:

Warning: Invalid size (3186400376) for InteropIFD tag 0x2

With this patch these warnings are gone (the interop is cut out).

Committed to DEFAULT.

Reported by oduis@hotmail.com on 2011-10-09 08:40:19

• Status changed: Fixed

[Beep6581]

Olli, thanks for this patch. With specifically Fuji files there is another problem -
the extra dynamic range for which fuji cameras are known for cannot be recovered by
dcraw/rt. I wonder if this is a bug or a lack of a feature?

Reported by michaelezra000 on 2011-10-09 19:00:17

[Beep6581]

The Fuji sensors like in the S5 have a different pixel pattern (non-bayer). Guess AMAZE,
DCB etc. won't handle that, so RT will rely on DCRAWs standard behaviour there.

Reported by oduis@hotmail.com on 2011-10-09 20:42:01

[Beep6581]

I see. The raw files get opened, but it seems that only a single set of pixels is used
and the other set with additional 4EV!!! of highlight data is ignored.

Reported by michaelezra000 on 2011-10-09 23:19:28

[Beep6581]

We would need a special code for this.  Questions:

1. Does RT at least load the data from the highlight pixels?  
2. Are the highlight pixels RGGB or just luminance (ie do they have color filters)?

Zeroth order I think would be to use the highlight data to extend DR.  More complicated
would be to use them to give higher resolution.  I've never looked at Fuji CCD raws,
I don't know what the possibilities are.

I'm also thinking that down the road, the code developed might be adapted for HDR along
the lines of Zero Noise HDR by Guillermo Luijk.

Reported by ejm.60657 on 2011-10-09 23:59:32

[Beep6581]

Here some info on Fuji CCD-s:
http://en.wikipedia.org/wiki/Super_CCD
http://www.dpreview.com/news/0301/03012202fujisuperccdsr.asp
http://www.color.org/CCD-honeycomb.pdf

I believe that both sets of pixels have color sensors.

I still have Fuji S3 (with really unbeatable dynamic range) and can of course provide
sample raw (RAF) files. Here is one I took of color checker: https://minus.com/mOMCFosTw
I will try to get some images with high DR. As a reference, ACR handles them very well.

There is a free S7raw converter which displays some mixing formulas for gaining resolution:
http://www.photo-freeware.net/s7raw.php

Reported by michaelezra000 on 2011-10-10 00:21:37

[Beep6581]

Sorry, what do they do for resolution?  There wasn't any info at the link, just software
to download.

Reported by ejm.60657 on 2011-10-10 00:37:22

[Beep6581]

Woah, this "raw fusion" to generate a raw from several raws is an amazing idea! The
benefits are immediately clear - not only for lowering noise, but also simply for not
having a mess of bracketed photos when they can be merged into one. I dream of the
day that I will be able to combine raws in RT! :]

Emil: from what I've read, those extra SuperCCD photosites share the same color filters
as the normal ones. RT does not read the extra ones.

Fun to see s7raw again, I used it for some time in the past when I had a Fuji camera,
but then I found RT :]

Reported by entertheyoni on 2011-10-10 00:45:14

[Beep6581]

ps. RT reads their sample merged raw file! Highlight reconstruction doesn't seem to
work at all, but the rest of the sliders do, although its clear that the sliders affecting
exposure and highlights aren't suited to working on files with such a high dynamic
range.

http://i.imgur.com/sdeWd.jpg

Reported by entertheyoni on 2011-10-10 00:59:44

[Beep6581]

@Emil, here are the screenshots from s7raw:
http://www.timelessme.com/temp/Postings/S7raw_SRmixing.jpg 

I never found a clear instruction for use of these mixings of S and R pixels.
I could make illustrations of various methods, if that helps (s7raw runs on windows
only)

Reported by michaelezra000 on 2011-10-10 03:27:57

[Beep6581]

@Slony: HL Recon is raw only; it won't operate on tiff output from some other converter.

@Michael: Thanks.  Looks like a variety of proposals about how to do the blend.

It seems that dcraw discards the highlight pixels.  We would need to get them loaded
into RT.  Fabio is the logical candidate if he is willing.  Apparently libraw has some
options for dealing with these pixels, perhaps that could be of some help.  After the
pixels loading code is in place I can try to use them for output.

Reported by ejm.60657 on 2011-10-10 03:46:20

[Beep6581]

It seem like FUJI has changed sensor structure in their current camera lines (the youngest
S5 with the sensor dicussed here is around 5 years old). They seem to be layered now
instead of having parallel highlight pixels.
http://photorumors.com/2011/04/20/new-patents-from-fuji-canon-and-hoya/
So it might not be worth the effort since it seems Fuji dicontinued that sensor the
pattern in their current models.

Reported by oduis@hotmail.com on 2011-10-10 06:32:37