Ideas for a default profile to produce predictable "camera-like" output

[Beep6581]

Originally reported on Google Code with ID 2071

Autolevels are today mainly used to make the first open of an image to look good (it's
used in the default profile). I'd prefer if first open should would just look similar
to a standard in-camera jpeg, ie just a tonecurve to brighten and compress highlights,
and that's it.

I've attached how such a tonecurve looks. Not sure if it can be achieved with the Exposure
box sliders. Try open a file, select the neutral profile (make sure highlight recovery
is also at 0 if you don't have bleeding edge when this is fixed), and then apply this
type of curve -- you'll see how very similar it becomes to the camera's JPEG output.

I think autolevels should not be included in any of the bundled profiles, and maybe
not even be saved into the .pp3 when enabled. Instead just seen as a settings "assistant",
you press once and get some values as a starting point for more advanced processing.
I could be convinced otherwise though, but I certainly don't think it's good to have
in the Default profile.

Why do I want this? Because ideally I'd like that you could dump all your raw photos
from a shoot on RT and do nothing and get predictable output. Autolevels is not good
to have in such a situation, it will try to correct an underexposed image, expand contrast
in low contrast conditions etc. A fixed curve (or corresponding) will give you something
that looks like normal "unprocessed" camera output.

It's also what basically all other raw converters do, default profile is something
stable similar to default camera jpegs. A few converters do have an "autolevels" function
(Lightroom has it for example), but it's not something that gets enabled per default,
as the results generally require further tuning by a human operator.

What do you think?

Reported by torger@ludd.ltu.se on 2013-11-20 19:25:54

[Beep6581]

My idea of Auto Levels is a tool that is supposed to get the darkest pixels to 0%, the
lightest to 100%, and to fiddle with the info in between to get it into the right ballpark.
Of course I can just move the bottom and top end points of a curve, or use the exposure
and black point sliders to do it, but why waste time when I can have RT do it for me?
Having the image in the correct ballpark, I can now use the other tools to fine tune
(how can you possibly use RT's Tone Mapping if you didn't first set proper black and
white points?) Likewise, bundled profiles can't possibly work well if the image is
not even in the right ballpark, which is why I think that for most of them Auto Levels
HAS TO be enabled.

The problem, however, is that currently I think auto levels is broken. Try with an
underexposed image. Jimr in the forum sent me [url=http://rawtherapee.com/forum/viewtopic.php?p=34280#p34280]two
severely underexposed raws[/url], and Auto Levels made the main histogram look pretty
much like the one you attached. Still severely underexposed! But now I'm hijacking
your issue.

I think there most certainly is a reason to use AL in bundled PP3s, as long as it works
properly!

Reported by entertheyoni on 2013-11-20 23:51:10

[Beep6581]

Oops, no bbcode here. URL above is:
http://rawtherapee.com/forum/viewtopic.php?p=34280#p34280

Reported by entertheyoni on 2013-11-20 23:51:42

[Beep6581]

And "Default.pp3" is supposed to be a good starting point for RT, not "As Camera JPEG.pp3"
;]

Reported by entertheyoni on 2013-11-20 23:52:37

[Beep6581]

One problem is that expanding the range to 100% is not always "right". I shoot winter
landscapes in dusk and dawn quite often, these nearly always get wrong with autolevels,
as the scenes have a natural low contrast. I don't do much portraits, but I would think
that autolevels is quite poor for portraits too, increasing contrast too much.

The very nature of an autolevels function is that it cannot make the right decisions
always, as it's scene dependent. Our autolevels is still better than say Lightroom's
autotone, which makes more aggressive settings but more often very wrong.

When deviating from how "all other" raw converters work we should also have a very
good reason. All other raw converters try per default to show something that's similar
to what a film camera would produce, ie also what default camera jpeg does. That means
a fixed tone curve and nothing more. Predictable output. Any user that has used any
other raw converter will get surprised when it's not possible to get camera-like output
per default.

I just don't see any strong reason why default should be a risky autolevels profile
with unpredictable output, and getting a camera-like output should require you to make
an own custom profile.

If people really want this to be default I'd still like to have an easily accessible
profile that gives this camera-like output. It could be called "Camera.pp3" or something
and should be right up there beside Default, so those of us that want "as usual" behavior
can get that without needing to learn how to do it and make our own custom profile.

During the first years using RawTherapee I have many times struggled with this issue,
that default profile do not give predictable output but instead add high contrast "pop"
through autolevels. I'm today using Lightroom when I need that, usually for high volume
work like a sports shoot when all processing I do from camera-like image is cropping
and perhaps post-crop vignette, and exposure when camera's auto-exposure is a bit off,
but no contrast changes or curves etc. Earlier I thought it was an advanced task to
make a camera-like output so I did not even try to make a custom profile. Now when
I've done more photo programming I know better, it's basically just a tone-curve. It
does tell me though that you need to be a pretty advanced user to know how to make
a camera-like profile, and that RawTherapee lacking this feature per default makes
it hard to use for common workflows.

Reported by torger@ludd.ltu.se on 2013-11-21 08:35:41

[Beep6581]

Note that I do appreciate RT's design where "neutral" (ie no slider changes) means raw-file
as-is only color space conversion, often looking flat, dull and a bit underexposed,
ie not really usable as-is but gives you good understanding of how digital photography
works and what changes needed to make a good image.

Commercial raw converters "neutral" means camera-like output, ie they have this tone-curve
per default, often without possibility to turn it off. To me that is being too user-friendly,
taking away control and understanding of digital photography from the user, and not
something RT should do.

However, we can still provide camera-like output by having a profile that applies this
type of tone curve, and I think it would be best that this is the "Default" profile,
but I can live with having it on the side at the top level.

I've introduced RT to users of other raw converters, and that the first open gives
an unpredictable and often "over the top" result (nowadays autolevels is a bit better
than it was though) is not something that is appreciated. People are used to start
with a "camera-like" image and continue from there.

Sidenote: With DCPs this tone-curve is typically embedded in the DCP and all you need
to do to get camera-like output is to apply the DCP, with tone-curve enabled. The intention
of DNG/DCP is to give camera-like output per default. I don't think we should do it
like that though, I don't like hiding away changes from the user.

Reported by torger@ludd.ltu.se on 2013-11-21 08:51:32

[Beep6581]

What do you think of this default profile?
I personally never use auto levels, since it engages RGB Contrast and it induces hue
shifts that ruins skintones.

Attached profile has two tone curves - brightening and darkening, set in thier preferred
curve types to the colors clean.

What could be useful is another, simpler, auto levels that only sets the exposure &
highlight recovery while user-selected tone curves handle the contrast. We could have
an option (in preferences?) - the Auto-levels intent.

Reported by michaelezra000 on 2013-11-21 12:57:30

---

• Attachment: Default-Curve1.pp3

[Beep6581]

The problem IMO is that each camera/sensor needs a different curve to make the output
look good. So any "default" curves will look good for some models and terrible for
others. My Pentax DSLRs e.g. need a very strong and strangely shaped contrast curve
(see attached) to make them look like the camera JPG output while my Canon S110 needs
a completely different curve and looks terrible when I use the "Pentax" curve.

The Pentax files also look horrible with RT's current default profile by the way...
I think there's no really good solution for this problem because sensor data is so
different. 

Reported by stefan.ittner on 2013-11-21 13:30:43

---

• Attachment: PentaxCurve.rtc

[Beep6581]

Yes I like the concept. I'll return with further comments, I shall do some experiments
based on your profile.

Reported by torger@ludd.ltu.se on 2013-11-21 13:45:06

[Beep6581]

Yes, I realize now that there's more to it than just a fixed curve to make really good
support. I'll try to nail down the requirements for a "minimum" profile which can produce
*stable* "camera-like"/"film-like" results.

Probably we need more than just a profile, ie some sort of autolevels, Michael's "autolevels
intent" idea could fit. A global default camera curve that can be refined with a model-specific
curve in camconst.json could also be a part of it.

I think autolevels could work mostly on exposure, push under-exposed files a little.
Then we may need to apply a curve modifier to heavily clipped files, so we get the
desaturated film-like overexposure look that camera files have.

Just speculating now though, need to make further testing.

Reported by torger@ludd.ltu.se on 2013-11-21 14:11:01

[Beep6581]

Sounds like a good idea. I think it's worth a look what Adobe uses as "base" or default
curves in Lightroom for different camera models.
I agree that Auto levels should focus on exposure and maybe highlight recovery. I also
think it should take into account the applied tone curves in its calculations (if any
are present), otherwise the result will vary a lot depending on what kind of curves
are used in addition.

Reported by stefan.ittner on 2013-11-21 14:26:54

[Beep6581]

I've worked a lot with Adobe's DNG reference code recently in another project, and got
a bit inspired by that. The DNG conversion is very basic, but still yields a stable
result which looks camera-like. Actually I have not seen any autolevels tricks in the
DNG reference code, it applies the curve and a baseline exposure and that's it. The
curve and baseline exposure can vary between models though, as they make a DCP file
per model.

The curve in my initial post forgot about gamma so it's a bit wrong, tuned the curve
to better match a typical tonecurve, and played with some with Canon 5D Mark II files,
and for that camera it seems like in most cases just that tone curve is enough to produce
a result very similar to the camera's own JPEGs. The Film-like curve (which is Adobe
DNG style) is a bit too saturated though, I get better results with RT's "weighted"
curve.

I shall provide a .pp3 later when I've played around some more. Need to look at pictures
with skintones in them as those are most sensitive.

Different camera models strive for different looks in their jpegs. To say something
general I think that DSLRs strive for a natural look in the default settings, while
consumer compacts often strive to exaggerate saturation to get more pop. Probably one
should strive to primarily achieve the natural/neutral camera-like look first-hand,
rather than mimicking JPEG look exactly (the cameras often have various look modes
too of course) as that will be lots of extra work.

All camera sensors that we support today are linear light capture devices, there's
nothing magical about them, so I think that it should be possible to make some basic
profile and possibly support algorithm that produces a stable result. The main difference
between models I think is that cameras auto-exposure devices may expose the sensors
a bit differently, ie how much highlight headroom they choose to keep. The DNG spec
solves this by having a per-model baseline exposure, but as said otherwise no dynamic
autolevels type of function.

Camera JPEGs cut away lots of highlight detail, as doing highlight recovery tricks
is dangerous in terms of getting stable results. I don't think any highlight recovery
should be enabled in this type of profile, but one should be just as wasteful (and
thus stable) as the cameras are themselves. Recovering highlights and make the most
out of your image is something that require manual work.

Reported by torger@ludd.ltu.se on 2013-11-21 15:35:03

[Beep6581]

Here's a curve that works very well with my Canon 5D Mark II, when I compare with the
cameras default neutral output. Unsurprisingly it's similar to Adobe Camera Raw's default.
I've set it to "weighted curve" here which gives a less saturated look. Many would
probably prefer the more saturated look of the film-curve. Probably the darks need
a bit fine-tuning.

I'm back to thinking that we can solve the camera look with 1) a fixed curve and 2)
a fixed baseline exposure. I think the curve can be same for almost every camera, while
baseline exposure can vary (fixed per model though). We could put curve and baseline
exposure into camconst.json. Adobe uses +0.4 stop baseline exposure for the 5D mark
II.

I tested a few other camera models and got good results with the same curve, but I
needed to have other baseline exposures, in the range +1.0 to -1.0.

Reported by torger@ludd.ltu.se on 2013-11-21 16:38:12

---

• Attachment: Default-Curve2.pp3

[Beep6581]

The results with your curve look pretty good. I guess it will give vastly superior results
for most cameras and pictures than the poor "default" setting. 

Funny, your curve has almost the same strange shape like my Pentax curve but has different
contrast distribution and a stronger lightening effect. Is there something special
about these "asymmetric S-curves"? 

Reported by stefan.ittner on 2013-11-21 17:10:11

[Beep6581]

The special thing with the asymmetric s-curve I think is that you brighten the image
substantially and compress the highlights. This gives a nice highlight rolloff towards
white without discolourings or strange transitions. "All" camera jpeg engines do that
because it's difficult to make use of the last highlight stop and get predictable behavior,
and you can't have a camera that fails randomly depending on the scene's subject.

To get this to work we would need
- add baseline exposure and curve to camconst.json, with hard-coded defaults (curve

  will match most cameras, baseline at +0 will work okay for many but need to be
  adjusted for many too)
- a feature in the pp3 which indicates that a default camera look should be applied,
  like autolevels work today
- apply curve and exposure at first load

There is curve and baseline exposure support in the DCP format, so one could have it
in the camera DCP too. In a way that is a cleaner solution, but requires more work
(ie DCP files for each camera), and I think it's nice that RawTherapee actually shows
the modifications to the user. I'm also not convinced that Adobe's film-like curve
is the best (in DCP it will be that curve, as that's what the DNG standard specifies).

Reported by torger@ludd.ltu.se on 2013-11-21 17:52:28

[Beep6581]

I tried Default-Curve2.pp3 seem to better apply to Canon 5DM2 than to Nikon raws.
I much more frequently use a 2-curve approach though to get better color purity.
We have many illustrations in the original issue 1513.

A camera-based tone curve is an interesting idea.
May be we could use the existing DCP tone curve, but extend it by supporting Curve
Type and Opacity?

Reported by michaelezra000 on 2013-11-22 04:05:12

[Beep6581]

Yes it would feel kind of boring just to copy Adobe's approach straight off, sure we
can do something better! :-)

Using up both tone curves in the profile might not be that good, as one may want to
use those for further adjustments. Possibly one could add a new optional curve and
exposure, and the "Camera Profiles" would then use those. It could be a small sub-box
at the bottom of the exposure panel which would be folded in the normal case.

Reported by torger@ludd.ltu.se on 2013-11-22 07:56:01

[Beep6581]

I'll read this thread tomorrow (it's late now), but as a first answer:

I've built the Deep Shadow profile to mimic as close as possible (and it's really close!)
my Pentax K-5 image rendering with the Vivid rendering selected in the DSLR.

So yes, it's very Camera dependant, and i've started to work on a way to make the RT's
image look like the DSLR ones by taking shots of my CC24 with all of my camera parameters
set to the lowest then the highest value, one after the other. By manually finding
a relation between each camera setting and RT's settings, and by the use of the CPB
that now have access of the metadata, we could end up with what we want here, and maybe
even get close of some in-camera cross processing settings. The work done for one camera
will also, generally, by applicable to the other models of the same brand.

Reported by natureh.510 on 2013-11-23 02:25:11

[Beep6581]

Two very different things are being discussed here:
(1) - A tool in RT that acts much like "Auto" Levels in GIMP/Photoshop - it stretches
the histogram so the darkest pixels are pure black and the lightest pure white.
(2) - A tool to get an image to look like the out-of-camera JPEG even if it remains
over/under-expose

(2) is not only camera-dependent but even camera-mode dependent. To have RT mimic a
camera it would also need to mimic all that camera's modes, such as "Portrait", "Landscape",
"Sports", "Fireworks", etc.

(1) is independent of cameras, and I would like RT to do this very simple thing. I
cannot make a profile which makes use of any exposure tool, such as tone mapping, work
well on images if RT isn't capable of getting the basic exposure into the right ballpark
first. And it currently has problems doing so.

This screenshot shows an 8-bit TIFF of blue_sky_tree_drslony_bkt1 using Neutral in
RT and a histogram of the same image in GIMP: http://i.imgur.com/5ZJt7PU.png
The same image with Auto Levels in RT and the histogram showing GIMP's Auto Levels:
http://i.imgur.com/bmlvhuI.png
RT left the image underexposed by over a stop (are the lines in the main histogram
meant to represent stops?)
RT was supposed to clip 0.02% of all pixels - this clip tool is broken, I wrote about
that in issue 1443. It seems to affect only the dark end of the histogram, not the
light end.

bressanone_ivy_wall.tif http://i.imgur.com/Gt9mAVw.png http://i.imgur.com/dsmUi1W.png
lagow_05_21_2_EV.tif http://i.imgur.com/Rp9Egnu.png http://i.imgur.com/q9LMfOR.png
(logarithmic histograms)

(1) would probably work if "clip" worked, and if AL didn't weigh the blacks as much
as it does the whites. And/or perhaps we need two Clip values, one for the blacks and
one for the whites?

Reported by entertheyoni on 2013-12-03 00:05:00

[Beep6581]

DrSlony, can you provide a RAW to test with? I can't confirm this with my D700 NEF files
(Neutral profile was underexposed with elder versions but got much better with one
of Torger's patches, same for neutral + AL)

Ingo

Reported by heckflosse@i-weyrich.de on 2013-12-03 00:20:56

[Beep6581]

Sure, http://rawtherapee.com/shared/test_images/

Reported by entertheyoni on 2013-12-03 00:29:36

[Beep6581]

Using bressanone_ivy_wall.pef, 0.02% of 10139904 is 2028 pixels. Image opened in GIMP
shows 0 composite white pixels, and 130 value white ones. Even though it converts the
image to 8-bit, a 2028 white pixel spike should be retained in the 8-bit image. If
I get GIMP head to compile tomorrow, I'll check in 16 or 32 bit per channel mode.

Reported by entertheyoni on 2013-12-03 00:53:21

[Beep6581]

Here's what I get using neutral + AL with blue_sky_tree_drslony_bkt1. http://www.i-weyrich.de/rt/issue2071_ingo.png

Ingo

Reported by heckflosse@i-weyrich.de on 2013-12-03 00:59:34

[Beep6581]

Same as mine: http://i.imgur.com/Q6JSSj4.png

Reported by entertheyoni on 2013-12-03 01:10:42

[Beep6581]

I think this image also illustrates well that the default output of RT is not very pleasing
but dull and unattractive. I bet the camera JPEG is miles ahead in terms of contrast
and brilliance. This is what I've found again and again through trial and error: It
is simply impossible to get really good colors and contrast by just using the sliders.
A simple tone curve as discussed above makes a world of difference and (if tailored
to the Raw sensor data of a specific camera) can give consistent and pleasing results
for most situations.
My suggestion would be to concentrate on building a small number of tone curves that
form the basic building blocks of profiles. A little variety in the curves is necessary
because different sensors need different curves to look good (e.g. my Pentax K-r/K-5/K-5
II DSLR shots can all share a tone curve that gives very pleasing results for all of
them but my Canon S110 compact camera needs a different curve). I think this is what
most people want: a good, consistent starting point that does not look very different
from the JPEG preview. And that is what is mostly missing in RT: We have lots of great
"artistic" and complicated profiles that aim for a specific look, but very little in
terms of "normal" options. Maybe it would be an idea to separate the bundled profiles
into "regular" and "artistic/special effects" in the future?
Anyway, for the future "regular" profiles I think 4-5 different tone curves as a basis
could already be sufficient for most situations. People can then choose the profile
that fits their needs (or sensor data) best and then only need to fine-tune their output
with the sliders (slightly increase/decrease contrast, add saturation/vibrance, adjust
black/white point).

Reported by stefan.ittner on 2013-12-03 08:56:48

[Beep6581]

#18: Let me give an example.

The DNG Converter embeds a thumbnail when it converts, this is not copied from the
original raw file, but instead rendered with the DNG ref SDK and DCP. This consists
of 1) baseline exposure and 2) tone curve, that's it. The tone-curve seems nearly always
be the same as "ACR Default", ie a asymmetric tone curve as discussed early in the
thread, but there are probably some exceptions. Note that having baseline exposure
reduces the need of different tonecurves! Ie I think if we add baseline exposure we
might not even need as much as five different tone curves.

This gives a nice default that gives a predictable result. It does not render exactly
as camera if the camera is set in a specific mode, but it renders very similar results
between cameras, ie a predictable look. With Adobe's film curve and somewhat saturated
matrices/huesatmaps it gives a specific adobe film-like look. This is also what Lightroom
does, ie it gives a "Lightroom" look, not the exact camera jpeg look, but it's predictable.

What I'd like to see is the same concept but for RawTherapee. A default "RawTherapee
look" which we decide what it should be.

"Camera-like" in terms of copy the exact camera output depending on its settings too
difficult I think, too much work and too little value, unless we make some smart look-and-embedded-thumbnail-and-find-parameters-algorithm
(similar to auto-distortion correct) but it would be hard to make robust and fast.
It was probably a mistake to pull the discussion to much in that direction.

The auto-levels is a separate function, and is like "auto tone" in Lightroom, I think
it should be treated as that. An autolevels *will not* render good predictable results,
it's scene dependent! Stretching the histogram *is not* what you want in many situations,
if you shoot a low contrast subject you don't want the histogram stretched. Many many
of my winter pictures look like crap with auto-levels.

Auto-levels should be something you press when you start working, not something that
you get rendered per default. If you just throw your files from camera in a directory
and render output without doing anything I want a predictable "rawtherapee look" not
random result depending on scene content.

Reported by torger@ludd.ltu.se on 2013-12-03 10:04:14

[Beep6581]

I agree that it would be too much work to mimic the camera output. Even if we neglect
the myriad settings and "looks" a single camera can offer and only concentrate on the
"default" look, it would be too much work as new camera models which can produce Raw
files appear almost every week.
So a predictable "Raw Therapee" look is what we probably should aim for. But I am not
sure we can achieve this with a single all-purpose tone curve, as my experience - for
my cameras at least - differs. However, I am not completely familiar with the term
"baseline exposure", does it simply refer to exposure compensation or is it something
else?

Reported by stefan.ittner on 2013-12-03 10:22:16

[Beep6581]

Baseline exposure:

It's the same as BaselineExposureOffset tag in the DNG specification.

It's a pure exposure setting (just as the exposure slider) which specifies how the
camera exposes images (with auto-exposure). Many cameras leave some extra headroom
of highlight detail and those will look under-exposed if you just apply the typical
tone curve. My Canon 5D Mark II has 0.4 stop baseline exposure in DNGs if I remember
correctly. Fortunately this seems to be a fixed constant regardless of camera setting
(ISO etc).

If you're on windows/mac I can recommend downloading Adobe's DNG converter and DNG
Profile Editor (it's for free) convert some images from different cameras and open
in DNG Profile editor and look around and compare. This shows the concept of how far
you can come with only a baseline exposure and a tonecurve.

Reported by torger@ludd.ltu.se on 2013-12-03 10:54:47

[Beep6581]

Thanks for the explanation!
"This shows the concept of how far you can come with only a baseline exposure and a
tonecurve" - maybe I'm wrong, but I thought Adobe uses slightly different tonecurves
for different camera models?
Anyway, if we manage to find a "magical" tone curve that works well for all cameras,
all the better! For evaluation purposes, I think we should test the candidate tone
curves with images from current models of all the main DSLR manufacturers (Canon, Nikon,
Pentax, Sony) plus some popular compact cameras that can produce Raw files like e.g.
Canon S110 or Sony RX100. I could provide Pentax K-5 II and Canon S110 shots. Maybe
one typical landscape and one portrait image from each camera should be enough?

Reported by stefan.ittner on 2013-12-03 11:26:22

[Beep6581]

Here is what I found with a quick search:
http://forums.adobe.com/message/5733105
"You can get the default ACR tone curve from the source code of the DNG SDK. Or use
dcpTool to extract the tone curve from whatever DNG camera profile you want. Note of
course that not all DNG camera profiles use the default tone curve." (...) "Most of
the newer profiles don't have the default curve; they in effect combine the look-up
tables with the curve."

Reported by stefan.ittner on 2013-12-03 11:46:09

[Beep6581]

I have not gone through all profiles, but you can with the DNG editor view them and
compare them with the ACR default curve in a diagram inside the program. Variations
are hard to detect with the eye on the profiles I've looked. So they may be slightly
different, but certainly not hugely different. Baseline exposure makes up for most
of the difference it seems.

Newer Adobe profiles does more tricks with combining extreme matrices with extreme
huesatmaps/looktables to reach the desired result. One can see this that by only using
the matrices the color becomes bad, but when applying huesatmaps/looktable it becomes
good. But this is when you use ForwardMatrices, when you use the ColorMatrices the
color is still sane, but it won't work with the huesatmaps. RT support these newer
profiles today by the way, but it does not mean anything for this issue, this is just
one method to arrive to some color rendition.

We don't have problems with color rendition here (apart from possible fine-tuning concerning
curve type etc), it's the right exposure and tone-curve we need.

The DCP curve can only work in one specified way, ie the same way as "film-like" RawTherapee
curves. The curve is only useful for controlling contrast and highlight/shadow compression,
not color rendering, on that part it just have the slight-saturation-increase effect
the film-like curve has.

Reported by torger@ludd.ltu.se on 2013-12-03 12:40:04

[Beep6581]

Anyway we can discuss this forever how different tone-curves need to be or not, without
really knowing. We need to do some tests to find out. One need to make a tone-curve
with "film-like" type which is similar to ACR default and make a profile with only
that (rest neutral), and test on various cameras and do adjustments only with the exposure
slider (=baseline exposure) and see how well it works.

I don't have as much time right now to work with RT so I will stay low on this for
a while.

Reported by torger@ludd.ltu.se on 2013-12-03 12:45:54

[Beep6581]

Only some thoughts, as I use a different workflow. I never use the default profile,
and the neutral profile only to build a new basic pp3 for a specific camera.

I made basic profiles for every camera I have, starting with the neutral profile. In
the exposure tab AL remains set to inactive. In Ciecam I made two cagecurves for lightness.
This profile is stored and then reused for making extended profiles for each isorange
(eg 100-400, 500-800, ...) where I define basic sharpening, noisereduction, basic contrast
in Ciecam if needed, .... These extended profiles are once more extended for each lens
(lensprofile) if necessary.

When I load a photo the specific profile is automatic loaded through the RT_ChooseProfile
script. This gives me a good basic image.

The only thing I wish is an automatic exposurecompensation instead of AL. I tried it
with AL and resetting all the sliders to 0 except exposurecompensation, and this gives
me a photo close to the jpg from the camera. This should give me the opportunity to
batch convert my images and giving me satisfactory photos for previewing also by others
(family, .... Albeit in a small (web)format).

Additional info: I use almost exclusively primes (Leica R lenses), shooting in full
manual mode (also for focusing. I have a Katzeye Ultrabright focusscreen for fast focusing),
so exposure is very stable and accurate in most circumstances.

Reported by reginald.ricquier@scarlet.be on 2013-12-03 14:02:38

[Beep6581]

Some cameras overexpose the JPEG by even 2EV, others underexpose by 0.3EV, relative
to the raw. There was an old page with quite a list of these values for different camera
models, can't find it now. If I understand what you're trying to do, you wont find
one universal baseline exposure.

This issue is a resurrection of an old issue where we bounced around the idea of making
JPEG-like profiles for each camera model. That didn't work out. I suppose if we make
a third profile category, so "My Profiles", "Bundled Profiles" and "Camera-Like Profiles",
and we populate this with the few camera models we have at our disposal, that will
start the ball rolling and people will help out. If we can extract these from DCPs,
all the better.

As for the "Auto Levels" button, I am against changing it's intention, which is to
get the histogram to touch both ends (or clip if we so choose, except that's currently
broken) and to make a good guess about what to do with the mid-tones. Issue 2104 will
get it working as it should. @ comment 4/25, most photos should make full use of the
histogram apart for shots in mist, overcast snow and such. Even stock photography websites
reject photos which don't comply with this. This is something which everyone usually
does anyway, it's just a time saver having a button move the curves and sliders to
the right place for you. If any new auto- button will be added as a result of this
issue (Auto Tone?), please treat it as an addition, not as a replacement for Auto Levels,
as their intended uses do not overlap.

Reported by entertheyoni on 2013-12-05 16:22:24

[Beep6581]

#33: "If I understand what you're trying to do, you wont find one universal baseline
exposure."
That wasn't the idea. Instead, torger's suggestion was to use a universal *tone curve*
while changing the baseline exposure based on the camera model. 
After playing around a bit with curves and changes in exposure I have to agree to his
proposal that one curve might already be sufficient (and not several as I suggested).
However, after some testing I find torger's proposed curve a bit extreme, at least
for the cameras I checked. It gives a very pronounced brightening effect, so most cameras
would probably need a negative baseline exposure correction. Also, as it's rather flat
on the right side, it has a bit too much highlight compression for my taste. 
I have attached a less extreme curve which works very well with my various Pentax DSLRs
and - with baseline exposure correction - also reasonably well with my Canon S110.
It mimics the Pentax JPEG output very closely (even without exposure correction), but
is slightly brighter and more contrasty than the camera JPEGs from the Canon S110.

Reported by stefan.ittner on 2013-12-05 19:49:01

---

• Attachment: [New Default Curve - Candidate 1.pp3](https://storage.googleapis.com/google-code-attachments/rawtherapee/issue-2071/comment-34/New Default Curve - Candidate 1.pp3)

[Beep6581]

Another thing: If RT's default profile will in future use a tone curve, it would probably
be necessary to modify the auto levels function. As it is, it does not seem to take
into account the presence of tone curves for its calculations. if it stays this way,
the results will probably be too bright because the tone curve will aready have some
brightening effect.

Reported by stefan.ittner on 2013-12-05 19:52:13

[Beep6581]

I guess AL would need to bypass setting the brightness and contrast, which is a valuable
option in itself, as they are using a standard type tone of curve which is not always
desirable due to hue shifts.

Reported by michaelezra000 on 2013-12-05 22:31:46

[Beep6581]

Still bringing back my idea of reading some Exif and/or Makernotes metadata like Baseline
Exposue, Contrast,Saturation, Detail, and few other ones. It can be either done through
the CPB or programmatically, the CPB being a more flexible solution.

Reported by natureh.510 on 2013-12-05 22:37:41

[Beep6581]

#33: Baseline exposure won't be universal no, that one can also see in the DNG converter
work, baseline exposure varies between models, while the tone-curve seems to have relatively
small changes.

I have not had time to fine-tune the shape of the tone curve, so my initial suggestion
rather rough.

I've attached "adobe camera raw default tone curve" here if you want to plot to see
their example. The first one is in linear gamma, the other mapped to RT's 569/256=2.22
space.

Reported by torger@ludd.ltu.se on 2013-12-06 06:48:02

---

• _Attachment: acr_default.txt_
• _Attachment: acr_default_g22.txt_

[Beep6581]

What do we do with these files?

Reported by entertheyoni on 2013-12-07 00:06:54

[Beep6581]

Re #34, Stefan: Your 'New Default Curve - Candidate 1.pp3' is too dark with my D200
and D700 files. I can provide files, if you're interested.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-01-10 20:40:24

[Beep6581]

It's too postcard-like on my K10D raws.

Reported by entertheyoni on 2014-01-10 20:55:48

[Beep6581]

#34: Yes, please provide a sample if you don't mind. 
As discussed above by Anders, the problem of too light or too dark can only be solved
by camera-specific baseline exposure values, like Adobe does. Otherwise it will be
impossible to find a curve that fits all camera models. So, the question is whether
the curve looks good (or better than the current default profile) _after_ baseline
exposure adjustment.

Reported by stefan.ittner on 2014-01-10 21:07:26

[Beep6581]

#41: "too postcard-like": do you mean too contrasty? I don't know the K10D, but for
the K-5 and K-5 II the result I get is pretty much identical to the default JPG setting
of the cameras as regards contrast. Could you please provide a Raw for testing?
Also, if you (or anyone else) can provide an alternative suggestion for a default base
curve, please do - it's would be helpful to have some alternatives for testing before
making a decision :)

Reported by stefan.ittner on 2014-01-10 21:15:25

[Beep6581]

Ok, here are two samples: 

http://www.i-weyrich.de/rt/D700_20120803_0369.NEF
http://www.i-weyrich.de/rt/D700_20121027_1393.NEF

Ingo

Reported by heckflosse@i-weyrich.de on 2014-01-10 21:33:05

[Beep6581]

Thanks! The first image opens very dark in RT compared to the embedded JPG - Auto levels
adds +2.48 EV in default profile! 
Hmm, difficult stuff. Nikon seems to use a very different approach to JPEG processing
compared to Pentax (or Canon). I can see that even after adding exposure compensation
the results with this curve are probably not going to be close to the embedded JPG.
Did you use default JPEG settings without shadow/highlight adjustments or dynamic range
enhancement?

Reported by stefan.ittner on 2014-01-10 22:07:28

[Beep6581]

I don't think that "camera-like" output is a correct generalized objective for the raw
converter. After all there is a reason why camera jpgs are not being used by raw-shooters;)

The curve indeed does look like a "terrible" camera output - I don't mean to say that
the work on the curve was terrible, just the camera output is;)

It would be much more useful to have default output which does not crush shadows and
does not burn highlights, yet maintains a pleasing global and local contrast and saturation,
while not causing strong hue shifts, and is fast!

The current default profile relies on the Auto Levels that is based on the Standard
curve's contrast that actually is prone to the hue shifts. Auto levels was the only
automated mechanism in RT when this profile was developed, and may be now is time to
improve the default rendering. The objective then was *to render image in a generally
useable way, to give a decent starting point*. Given the hue shifts, I cannot see the
current AL function as a usable starting point for professional-level rendering.

If we decide to use a constant curve instead of adaptable AL, I think we should be
careful to not use curves with a strong contrast.

Reported by michaelezra000 on 2014-01-10 22:19:50

[Beep6581]

Here's a comparison for a Pentax K-5 II file, without any exposure adjustments necessary
for the new curve:
http://filebin.net/t90p364z0p 
- Camera JPG
- New curve
- RT Default

The camera JPG with factory default setting was re-created with the Pentax software,
the embedded preview in the DNG used the less colorful "natural" JPG setting.
(White balance would need to be corrected, camera's AutoWB is too much magenta)

Reported by stefan.ittner on 2014-01-10 22:21:53

[Beep6581]

re 45: At the time I got my D700 some years ago I set it to RAW and didn't care about
JPG. But that doesn't mean that I didn't change some JPG-settings in camera by accident
meanwhile, don't know.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-01-10 22:32:15

[Beep6581]

Most of the images here are from my K10D: http://rawtherapee.com/shared/test_images/
I took some raw+JPEG in-camera shots recently for the purpose of this issue, so I can
provide a curve that gets RT's rendering looking like in-camera for the K10D, after
the weekend.

Reported by entertheyoni on 2014-01-10 23:01:31

[Beep6581]

Perhaps that's a good way: To provide one default-curve per camera model? But how about
different lenses with different contrast? I think, it will be very difficult and time-consuming
at least for the devs to get this thing right and I also think, that the the actual
default profile is not too bad (it's in fact pretty good, I think). My vote goes for
letting this Issue open. If someone can provide a good curve for his cam, why don't
include it in bundled profiles or wherever?

Ingo

Reported by heckflosse@i-weyrich.de on 2014-01-10 23:15:55