Closed Beep6581 closed 8 years ago
A hierarchy of and some notes on Super CCD sensors
Super CCD # of images CFA HR mode SR mode SR ev. SR mode
Camera models
| in raw file @45° img.@45° img.@45° diff. shift
(supported by dcraw)
|
o- Super CCD / Super CCD HR 1 yes yes no 0 yes
S1Pro S2Pro E550 E900 F800 F810
|
S5000 S5100/S5500 S5200/S5600
|
S6500fd S7000 S9000/S9500 S100FS
|
o- Super CCD SR
| |
| o- Super CCD SR 2 yes - yes 4 no
S20Pro F700 F710
| |
| o- Super CCD SR II 2 yes no yes 4? yes
S3Pro S5Pro
|
o- Super CCD EXR / CMOS EXR 2 no yes no 0-2 yes
X10 XF1 XS-1 IS-1 S200 HS20
HS30 HS50 F550 F600 F770 F800
HR = high resolution, SR = super dynamic range (or low noise), these images have half
the number of pixels of an HR image and there are always two of them for one HR image,
each containing every other pixel of the HR image.
Atleast theoretically for most sensors, HR and SR modes could be simulated in a raw
developer when needed.
All sensors use a CFA with a Bayer pattern for each image they produce, they differ
in the number of images they produce and in the orientation of the CFA, so for some
sensors HR mode images are rotated 45° but SR mode images are not and vice versa for
others.
The first SR Super CCD had a low and a high sensitivity photodiode on every photosite,
thus there is no visible shift between the two images this sensor produces, trying
to construct a HR image from that is useless. Later models have two different types
of photosites each with just one diode so, there is a visible shift between the two
images and a HR image can be constructed.
when rotating an image and doubling the number of pixels, bilinear interpolation can
be used to fill in the empty pixels, like dcraw's fujiturn does, maybe another (AHD-like)
interpolation method would produce better results.
When combining two EXR images, with the same exposure, by simply binning pixels, the
amount of noise is reduced considerably but so is the sharpness, maybe some other method
could achieve better results. The same probably holds true for the Super CCD and Super
CCD HR sensors, when rotating and shrinking the image at the same time.
When combining Super CCD SR II images or EXR images shot in DR mode into an HR images
it maybe necessary to reduce the amount of noise in the darker image first.
Reported by frank.markesteijn@gmx.com
on 2013-08-22 13:50:28
We should consider the pre-demosaic-steps like 'auto CA-correction', 'Hot/Dead-pixel
filter', 'cfa-line-denoise' etc. when thinking about class-hierarchy...
Ingo
Reported by heckflosse@i-weyrich.de
on 2013-08-22 21:45:24
Good point Ingo, about the pre-demosaic steps.
Super CCD sensors and four colour sensors too, are very much like Bayer sensors here
but, X-Trans is a little bit different. For CA-correction in the former you could use
bilinear interpolation to calculate the new values, without creating a zipper artifact
in the colour plane being scaled, for X-Trans that will create the zipper effect, you
will need something more sophisticated to avoid it. If same coloured pixels are extracted
from a raw image and combined in a half sized image, for CA-correction or denoising,
that can be done without changing their orientation toward each other, for X-Trans
that is not possible.
One more note on Super CCDs
Creating HR images from two EXR (or SR) images can be done after demosaicing but, it
could also be done before, creating a raw image with yet another RGB-pattern, the X-Trans
demosicing algorithm can be adapted to handle that. Will it be any good? I don't know.
Frank
Reported by frank.markesteijn@gmx.com
on 2013-08-25 10:45:58
I think, first step should be to support non-Bayer-sensors for demosaic, If any of the
pre-demosaic-steps doesn't match to the sensor, it should be disabled until we have
a solution.
Ingo
Reported by heckflosse@i-weyrich.de
on 2013-08-28 23:11:27
Reported by natureh.510
on 2013-09-18 23:39:06
Reported by natureh.510
on 2013-09-23 16:24:03
I did some research about the non-Bayer formats:
Here's a first link to Fuji X-trans: http://www.dmcgaughey.com/2012/11/07/fuji-x-trans-raw-conversion/
Maybe we should collect informations about this formats before trying to develop an
algorithm to process the matrix.
Ingo
Reported by heckflosse@i-weyrich.de
on 2013-09-24 21:41:13
Frank, have you started any patch or looked at the code to see the problem we should
solve?
I'm currently looking at the class hierarchy, and would like to talk about the EXR
sensors, as explained here: http://finepix.com/exr_cmos/en/
- On the HR image, what can we do with this kind or arrangement? I see 2 interleaved
images, shifted diagonally, but not rotated matrix...
- On the DR image, we have 2 interleaved bayer matrix, shifted diagonally; could we
consider those images not shifted (and then with a reduced size) and simply fuse them,
or should we use more complex math to fill the hole on both image then fuse them?
- On the SN image, we could considerate the matrix as 2 interleaved standard bayer
matrix, shifted diagonally. Could we use simple pixel binning here with reduced image
size or should we have more complex math?
In order to fuse image 1 and image 2 from the same raw file, i think it would be time
to generalize the image fusion concept to bracketed images too. I'm thinking in this
direction and already made some test with RT+Gimp with quite nice results. I'll open
a new issue for this, so support for dual image may be delayed but not X-Trans implementation,
which is the more urgent IMHO.
If you think that we should keep it simple and code dual image support ASAP, just say
it.
Reported by natureh.510
on 2013-09-24 23:08:07
Started
Just my small addition. I cant speak for everyone, but most converters which produce
6 mpix output out of S5 and S3 Pro give rather sharp and nice results.
Those converters which produce 12 mpix files are always bit unsharp due interpolation
(filling gaps). As that resolution really isnt there, it never looks "right" to me.
Both usually have certain amount of artifacts, tho Im quite positive 6 mpix could be
made to be almost artifacts free.
Im using LightZone right now, which uses DCraw to get 6 mpix files from S5/S3 Pro and
then its fused together (I guess). Issues is that it uses some kind of demosaicing
algorithm which produce rather nasty jaggies (borders between light/dark are usually
visible .. 100 - 200% view). And hot pixel removing is rather poor..
If that was solved (maybe by some variant of AMaZE), then it would be probably closest
to perfect.
Other one is RPP for Macs which uses modified variant of VNG and AHD, but both are
prone to cause moiré with these sensors, so probably not best idea.. (not exactly sharpest
either).
Btw. there is already software which uses both files from S5/S3 Pro to create resolution,
its SilkyPix. It has nice output, but it still looks like stretched lower resolution
(but its smooth I give them that, mostly no issues with jaggies or so).
Im for simply fusing those 6 mpix files, preferably with some really good demosaicing
algorithm which will prevent both jaggies and moiré. Not sure my vote counts, just
my .5 cents.
Yea and I would really love to see Sigma supported, especially with color processing
possible inside RT, that could fix most of SD1M (and similar) color issues. :)
Reported by Mescalamba
on 2013-09-25 04:28:15
In response to Ingo's remark, post #7
Unfortunately, if you want to demosaic something other than a Bayer CFA, you're on
your own, there isn't any usefull information to be found, like scientific articles,
almost a year ago I searched every crevice of the interweb for it and came up empty
handed, some people have published their search for an X-Trans demosaicing algorithm,
I didn't like the results and came up with something different, after months of experimenting.
In reply to #8, Jean-Christophe's post:
I have not yet started any patch for Rawtherapee, I have been working on an HR demosaicing
algorithm for EXR (and Super CCD SR II).
HR: EXR raw files contain two normal Bayer images, combining them produces a rotated
image.
DR: That shift can be ignored.
SN: Just bin the pixels.
Demosaicing an EXR image after combining the two raw images produces better results
than combining two demosaiced images but, if you then reduce the size of the image,
the results is no better than binning the pixels of two demosaiced images, ofcourse
they should be binned before demosaicing but, I haven't tried that yet.
bracketed images: The two EXR images can be treated like any other set of bracketed
images.
I think EXR support is as desirable as X-Trans support but, more work because, among
other things, two images need to be loaded and DCRaw wasn't made to do that. It would
be nice if DCRaw could load two images from one file, other projects could benefit
from that too but, it would not help in loading bracketed images. So should we code
dual image support ASAP? If we could convince or help Dave to support it in DCRaw I
think that would be preferable.
And in reply to Corwin's post, #9.
The S3 and S5 have Super CCD SR II sensors, and, contrary to EXR, both images in a
raw file are rotated (conceptually), just like every other Super CCD image, the combined
image is straight, those 6mp images have been rotated not the 12mp ones. When I combine
the two images I get somewhat better resolution than in a rotated 6mp image from DCRaw
but, because the second image is almost four stops underexposed it is noisy, when you
supress the noise you loose the extra resolution. As for artifacts, I get a bit more
moiré in 12mp images than with EXR but, 6mp images can be demosaiced with any Bayer
algorithm, I don't see more or less artifacts in them than in other images. Should
RT support an HR mode for Super CCD SR II? Once EXR is fully supported, adding full
support for the S3 and S5 is easy, you need the same algorithms, you just apply them
at different times.
Reported by frank.markesteijn@gmx.com
on 2013-09-26 17:41:00
Maybe it's not clear how or why two perfectly normal
images are combined into a rotated one.Hopefully the
drawing answers the how. Why? Because it's the only
way two images can be merged while preserving all
pixels and without leaving gaps or creating any
distorsion, other than a rotation and a scaling.
_____ _____ _____ _____ _____ _____
| | | | | | | |
| A | A | A | | B | B | B |
|_____|_____|_____| |_____|_____|_____|
| | | | | | | |
| A | A | A | + | B | B | B | =
|_____|_____|_____| |_____|_____|_____|
| | | | | | | |
| A | A | A | | B | B | B |
|_____|_____|_____| |_____|_____|_____|
_____ _____ _____ _____ _____ _____
| | | | | | |
| | | A | B | | |
|_____|_____|_____|_____|_____|_____|
| | | | | | |
| | A | B | A | B | |
|_____|_____|_____|_____|_____|_____|
| | | | | | |
| A | B | A | B | A | B |
|_____|_____|_____|_____|_____|_____|
| | | | | | |
| | A | B | A | B | |
|_____|_____|_____|_____|_____|_____|
| | | | | | |
| | | A | B | | |
|_____|_____|_____|_____|_____|_____|
Reported by frank.markesteijn@gmx.com
on 2013-09-26 19:19:07
Corwin,
Maybe those programs merge the S and the R pixels and then rotate and magnify the resulting
6mp image, that would explain the lower quality, I asumed those programs would create
an HR image for 12mp output, they probably don't. Sorry.
Frank.
Reported by frank.markesteijn@gmx.com
on 2013-09-27 12:15:46
No problem.
I think that S+R and magnify is what ACR/LR does, otherwise I dont know why it has
such miserable quality at 12 mpix.
I tried newer (bit more stable) version of Photivo, which outputs 6 mpix and allows
you to select demosaicing patterns you wish (I think almost all which are in RT too).
Results are rather nice. Its freeware too. Maybe it would be helpful to find out how
it works.. but up to you. :)
Reported by Mescalamba
on 2013-09-27 17:33:34
Update
No patches for Rawtherapee yet, but the EXR demosaicing algorithm is ready and making
dcraw load two images from one raw file turned out to be easy, very little code is
needed. I have not yet made dcraw produce high resolution images directly, at the moment
I am trying to make it produce "HDR" images from S3Pro and S5Pro files but, there are
strong colourcasts and I don't know where they come from, wether or not that is a problem
for EXR images in SR-mode too, I don't know, I have not been able to find any on the
internet, if someone could make some samples available, I would be gratefull. There
are many EXR raw files available for download but, in all, both images have been exposed
identically, I am looking for raw files where the second images has been underexposed
one or two stops, that is SR 200 mode or SR 400 mode pictures.
I did get dcraw to produce low noise pictures from EXR raw files, combining the two
images in a raw file before demosaicing does not just reduce the amount of noise it
also reduces the amount and intensity of demosaicing artifacts, more than overlaying
two images after demosaicing, in an image editor. The image does get blurred slightly,
in all respects, it is as if a stronger aa-filter has been applied.
I hope these changes will make it into dcraw, that would benefit many free software
projects not just Rawtherapee.
Bye, Frank.
Reported by frank.markesteijn@gmx.com
on 2013-10-15 12:46:10
Frank, if it is possible to communicate with s7raw authors .. they could solve many
problems regarding exr files (s7raw includes support for F200EXR..)
Reported by iliasgiarimis
on 2013-10-15 13:41:32
I mailed the author, lets hope he has an answer.
Reported by frank.markesteijn@gmx.com
on 2013-10-15 16:24:15
Lightzone has S5 Pro support too, currently they are at beta of 4.1 based on latest
DCraw 9.19. Not sure about F200EXR (depends most likely on DCraw). Tho they are not
exactly over-sharing in any sense..
Reported by Mescalamba
on 2013-10-15 16:45:40
Re #14:
That sounds great! :) Do you have any resulting image to share?
I thought that both images had to be worked on before merging them? At least, that's
what i did: I extracted each image from the RAF file (the patch for this test is straight
forward), reworked them separately, then merged in Gimp with quite a nice result.
If you want to merge both images before demosaicing, that shouldn't be a problem in
rawimagesource... But is it what we want?
Reported by natureh.510
on 2013-10-17 00:44:12
In case of fuji s3 there are distinct 12 million pixels. It would be great to extract
more than 6mp resolution from it's RAF files.
Reported by michaelezra000
on 2013-10-17 01:11:22
There's also black and white cameras such as Leica m monochrom and phase one iq260 achromatic,
and ir modified cameras. This is quite simple to get working on current code base.
Reported by torger@ludd.ltu.se
on 2013-10-26 18:31:31
Everything has been implemented in a modified version of dcraw, I shipped it to Dave,
we'll have to wait and see what he thinks of it, meanwhile here are some samples:
A 12 Mp HR-mode image of some backyards shot with an S5Pro, the original can be found
here:
http://www.rawsamples.ch/raws/fuji/s5pro/RAW_FUJI_S5PRO_V106.RAF
The command I used to create the tiff file:
dcraw -w -o 1 -q 3 -H 5 -s hr -m 3 -6 -T RAW_FUJI_S5PRO_V106.RAF
The 16-bit tiff: https://drive.google.com/file/d/0B421LKMHeJKNRUZHVXdDZzdiZWc
And a jpeg file made with Rawtherapee from the tiff file:
https://drive.google.com/file/d/0B421LKMHeJKNOTB6WHludHJiMDA
A 6Mp SR-mode image of a mountain, with a large dynamic range, shot also with an S5Pro.
Original: http://michaelezra.com/Projects/RT/images_shared/CO_2006_D8_556.zip
Command: dcraw -w -o 1 -q 3 -s sr -m 3 -6 -T CO_2006_D8_556.RAF
Tiff: https://drive.google.com/file/d/0B421LKMHeJKNc3BGWVVBSHRwRlE
Jpeg: https://drive.google.com/file/d/0B421LKMHeJKNd2lqVEZGcFQ0dzg
And the HR-mode version:
Command: dcraw -w -o 1 -q 3 -s hr -m 3 -6 -T CO_2006_D8_556.RAF
Tiff: https://drive.google.com/file/d/0B421LKMHeJKNejVQdTlEZV9kcTA
Jpeg: https://drive.google.com/file/d/0B421LKMHeJKNR0pRd05QRjNiTGc
A 12Mp HR-mode image of the dpreview test setup shot with an X10.
Original: http://movies.dpreview.com.s3.amazonaws.com/fujifilm_x10/DSCF0264.RAF.zip
Command: dcraw -w -o 1 -q 3 -s hr -m 3 -6 -T DSCF0264.RAF
Tiff: https://drive.google.com/file/d/0B421LKMHeJKNSTE3eXo1RFRyb1E
Jpeg: https://drive.google.com/file/d/0B421LKMHeJKNeFVGNFdfZjN0SEE
A 3200 ISO image of a street and la sagrada familia in Barcelona, X10.
Original: http://img01.quesabesde.com/media/img/noti/0055/fuji_x10_dscf0210.raf
Regular 6Mp image:
Command: dcraw -w -o 1 -q 3 -T -6 fuji_x10_dscf0210.RAF
Tiff: https://drive.google.com/file/d/0B421LKMHeJKNWlRZc1JrV0dkamc
Jpeg: https://drive.google.com/file/d/0B421LKMHeJKNQWUzeGpMZ2RXREE
SR-mode version with reduced noise:
Command: dcraw -w -o 1 -q 3 -s sr -j -T -6 fuji_x10_dscf0210.RAF
Tiff: https://drive.google.com/file/d/0B421LKMHeJKNYTRTMURiS1FLUVk
Jpeg: https://drive.google.com/file/d/0B421LKMHeJKNM0QweUZ2cXBob28
A landscape with a bridge and a boat, X10.
Original: http://www.imaging-resource.com/PRODS/X10/FULLRES/YDSCF8476.RAF
Regular 6Mp image:
Command: dcraw -w -o 1 -q 3 -H 5 -m 2 -6 -T YDSCF8476.RAF
Tiff: https://drive.google.com/file/d/0B421LKMHeJKNZjNzaU9sejh1Skk
Jpeg: https://drive.google.com/file/d/0B421LKMHeJKNYlUyd3E3aDZtSlk
SR-mode:
Command: dcraw -w -o 1 -q 3 -s sr -H 5 -m 2 -6 -T YDSCF8476.RAF
Tiff: https://drive.google.com/file/d/0B421LKMHeJKNNkh0NmpkdW1CbEk
Jpeg: https://drive.google.com/file/d/0B421LKMHeJKNcnBFcGs0N1c0Y2c
HR-mode:
Command: dcraw -w -o 1 -q 3 -s hr -H 5 -m 2 -6 -T YDSCF8476.RAF
Tiff: https://drive.google.com/file/d/0B421LKMHeJKNQ2JkSWMwZEp3M00
Jpeg: https://drive.google.com/file/d/0B421LKMHeJKNM2YteU9zVGwwSDg
Reported by frank.markesteijn@gmx.com
on 2013-10-28 20:49:30
I've worked on the mountain image to see how RT could process it and merge the subimages.
I must admit that I've never been close to what you've achieved here. Really nice work,
and can't wait to see it included in dcraw and in RT!
On a side note, I'd really like to know how the S5 compete to nowadays DSLR, like my
K-5 or other expert or high end cameras, but on the same scene, of course, otherwise
there would be no comparison possible!
Again, "chapeau" for this improvement :)
Reported by natureh.510
on 2013-10-28 22:43:35
S5 Pro cant compete with today dSLRs. Only in DR. That mountain image is from S3 Pro.
Otherwise that HR from S5 Pro looks quite a lot like SilkyPix output (except SilkyPix
is a LOT smoother).
There is reason why I use my S5 Pro only in 6 mpix, one of them is that resolution
simply isnt capable of full 12 mpix like regular camera. Tho I will be really happy
if there will be 6 mpix mode just with that extended DR. :)
Anyway. Im glad there is some improvement. Thank you for your effort.
Reported by Mescalamba
on 2013-10-29 05:01:21
Is it actually possible to communicate with Dave Coffin? I've mailed a trivial one-off
mem access fixes for Sony ARW2 parsing a couple of times, and some other minor stuff
and never got any reply, and the fixes has not been incorporated. Now I've found a
bug for the Kodak .DCR parsing, but it seems meaningless to report it back. Could be
that my mail address is getting into his spambox or something though.
Reported by torger@ludd.ltu.se
on 2013-10-31 09:43:07
I think your mails are getting through, it's just difficult to get his attention, and
that isn't motivating people, I think it would be better if it were run like a regular
project with a repository, buglist, discussions, more contributors and that formats
would be documented while being reverse engineered, because if Dave ever stops we have
a big problem.
Reported by frank.markesteijn@gmx.com
on 2013-11-01 12:47:20
Maybe this 12mp image of the mountain looks more like Silkypix' output, created with
dcraw, netpbm and my own version of fujiturn.
Command: ~/dcraw/dcraw-9.19 -w -o 1 -q 3 -m 1 -s sr -j -6 CO_2006_D8_556.RAF
Tiff: https://drive.google.com/file/d/0B421LKMHeJKNc2xNaXpyb2loOUU
Jpeg: https://drive.google.com/file/d/0B421LKMHeJKNeXVtaTJMUW5TVnM
The difference with the previous 12mp version is that this one has a wider dynamic
range and half the pixels have been interpolated. HR-mode does not do any interpolation
after demosaicing for S3 and S5 images, for the backyards I think that worked, for
the mountain view this new version is probably better.
Reported by frank.markesteijn@gmx.com
on 2013-11-01 19:57:06
Lots of jaggies, but overal improvement.
https://www.dropbox.com/sh/mws6fm247f8xe5l/Jln64cS-_j
This link contains one RAF file from my S5 Pro (you can use that image for anything
you want). Its I think good example for testing DR and testing that rather annoying
magenta tinting in blown-out highlights (well blown out at least one channel).
There are three JPGs in full size (you need to use blue Download to get originals Im
afraid). They are made from that RAF file provided. Two of them are made by Photivo,
no tweaks apart from incredibly small minus exposure correction. Photivo can use color
channels either like any other converter (thats function "intensify", you use Green
channel as base, amplify rest) or it can use colors way which simply uses most exposed
color channel as base and rest is reduced to maintain proper ratio. Obviously it does
have effect on higlights.
One JPG is from SilkyPix (converted to JPG via RT), pretty much default, just tweaked
to fit DR into proper space, lowering contrast and bit of negative EV was enough.
You can tell files apart by file names.
Last two are exmple of that S3 Pro shot of mountain. One made with SilkyPix (RT conversion
to JPG), one with another free to use converter, LightZone. IMHO one of best when it
comes to DR in S3/S5. You can choose there, where you want to blend those two sensors.
In this case it was blend at +2eV. No other tweaks. Demosaicing isnt perfect tho, but
decent.
As you can see, both LightZone and Photivo are 6 mpix converters. Im using Photivo
for all my S5 Pro files that I shot "correctly". Simply cause their variant of AMAZE
with EECI is one of best detail squeezing algorithms I saw yet. And it does reproduce
S5 Pro files properly most of the time. Plus it has some really good features. LightZone
might give a bit of edge in DR, but rest is too poor to be used or too clumsy. SilkyPix
12 mpix output is simply meh. I prefer sharp 6 mpix files over undefined fuzzy 12 mpix
with interpolation.
In case you need more RAF files from S5 Pro, I can supply them. Just tell me what kind
you want and I will either find it or shoot it.
Reported by Mescalamba
on 2013-11-01 23:52:11
Btw. issue with that S3 Pro mountain shot is that 1) it isnt exposed right 2) it really
isnt sharp photo by any stretch of imagination. Tho as DR testing pic its decent. Unfortunately
I dont own S3 Pro, so cant remedy that.
Reported by Mescalamba
on 2013-11-01 23:53:49
Issue 1212 has been merged into this issue.
Reported by torger@ludd.ltu.se
on 2013-11-14 15:55:11
Issue 2033: a patch to make fuji x100 (and possibly other) black levels be parsed correctly.
I hope it doesn't mess up for other cameras. I'd like to have some testers on that.
Reported by torger@ludd.ltu.se
on 2013-11-14 17:44:42
Reported by entertheyoni
on 2014-04-08 10:59:03
Reported by entertheyoni
on 2014-04-08 11:00:34
Reported by entertheyoni
on 2014-04-09 08:54:00
Reported by entertheyoni
on 2014-04-24 15:08:29
dcraw actually has interpolation for x-trans and foveon sensors. As soon as my actually
open issues are completed, I'll try to integrate these interpolations into RT.
Ingo
Reported by heckflosse@i-weyrich.de
on 2014-05-24 21:35:04
Hello! I'm curious what the status is of this. As far as I can tell, dcraw has not incorporated
the proposed changes (my version doesn't, anyway; the option -s sr produces the same
output as the option -s 0). I use a Fuji X10 and recombine two exposures obtained from
dcraw (with the option -s all) into a TIFF that I then import into RawTherapee. Would
it be possible to patch RT to combine the exposures, rather than dcraw?
Reported by john.perry@usm.edu
on 2015-04-16 21:39:32
Concerning FujiExr there's nothing done yet. Fuji x-trans support was added with 4.2.
Actually we're working on Foveon support.
Ingo
Reported by heckflosse@i-weyrich.de
on 2015-04-16 22:15:06
Originally reported on Google Code with ID 1944
Reported by
natureh.510
on 2013-08-11 13:38:17