Closed Beep6581 closed 6 years ago
Please take a look at the illustration of the curves concept:
http://filebin.net/t90p364z0p/RT_default_001.jpg
Reported by michaelezra000
on 2014-01-11 00:21:44
A pretty good rendering actually can be made with that S-type Film-like curve and alowered
contrast:
http://filebin.net/t90p364z0p/RT_default_002.jpg
The drawback is that it take more time to fine-tune this curve than the dual curve
model.
Reported by michaelezra000
on 2014-01-11 00:28:33
Michael, by saying 'one curve' I didn't mean 'one curve', but one different sets of
curves or whatever settings per camera...
Ingo
Reported by heckflosse@i-weyrich.de
on 2014-01-11 00:30:05
Here is one more illustration of a dual-curve rendering:
http://filebin.net/t90p364z0p/RT_default_003.jpg
Ingo, yes I understood your suggestion:) I think that could be a DCP profile curve
Reported by michaelezra000
on 2014-01-11 00:40:10
#50: Considering the apparently very different behaviour of Raw files from different
cameras with a fixed curve, I have to concur. I like the idea of bundling curves for
specific cameras, maybe in a special folder of the profile selector.
Reported by stefan.ittner
on 2014-01-11 19:29:07
The easiest way - RT already has an automation for camera-model-based selection of dcp
profiles. The DCP profile can have an embedded curve, which is actually a Film-like
curve (Olli modeled it based on Adobe's specs). One could simply use the camera DCP
profile with a curve and enable/disable curve with the existing checkbox.
Reported by michaelezra000
on 2014-01-11 19:56:25
#56: Yes, that would be the most elegant solution.
About your curve examples: I am not sure what you want to demonstrate? The different
curve algorithms?
For skin tones, in fact I usually use a mix of curve types because film-like and standard
give too much saturation/warming effect for my taste and weighted standard alone often
looks too bland, even with added saturation. By the way, wouldn't it also be nice to
have the selection of algorithms also for the contrast slider?
Reported by stefan.ittner
on 2014-01-11 21:49:08
+1 for selecting algorithm for contrast (and lightness and others)
If I understand correctly all sliders together are defining a curve in the end. So
we have (up to) three consecutive curves applied but the first (the one defined by
the sliders) is invisible .. well .. I would like it visible and to have the option
to change it visually ..
Reported by iliasgiarimis
on 2014-01-11 23:05:59
It would be incredibly useful to have selection of algorithms for the contrast slider!
The dual curve approach - separation of darkening and brightening curves has advantage
over a single S curve in that it is much easier to control to get the desired effect
as
1. range of motion for each points on the curve is wider
2. color shifts can be avoided much easier
3. independent curve types allow differential color rendering of brightening and the
darkening effects
Reported by michaelezra000
on 2014-01-11 23:11:58
At this point,would it be possible to have a graph visualizing the actual effect of
the contrast slider and its different algorithms as a curve?
Given that the lightness and contrast sliders are in fact applying a curve to the image,being
able to see which kind of curve manipulation is actually happening when moving the
slider would be very interesting and educative IMHO.
Reported by msth67
on 2014-01-12 10:09:34
A reliable default profile is one of the most important thing for users who are not
experts in RT, and considering the ever growing complexity of tools in RT this group
may be quite numerous.
The best way is most probably the one profile per camera.
Our workflows may differ, but when I compare two images, I check areas pairwise for
color tone, shadow depth, color casts, and so on. I think this process could be implemented
as an algo to get a roughly matching curve for every camera, maybe this could be even
built in into RT as a "match the colors of two images" tool.
The user marks areas (like in hugin, but here small areas) on two images as matching
areas, and the algorithm builds curve(s) that make the colors as close in the areas
paired as it gets. The implementation may struggle with contrast, but again, this would
be a rough base curve generator.
In my custom default profile for the ol' trustworthy D200 these are the major players
(related to this topic):
- auto exp off, all sliders at neutral
- two weighted saturation curves, one for shadows, one for highlights, these determine
the contrast too
- prophoto color space
For fine contrast tuning I prefer the lab L curve, and fine color tuning is done with
the HSV Eq.
The D200 has remarkable skin tone rendering at ISO 100-320. It looks like a slighty
unsaturated image, although it isn't: it's just beautifully balanced. Honestly, I've
spent days trying to figure out how to nail it in RT without consistent success.
Sometimes I can, sometimes I can't, and most probably it is related to how sensitive
RT is to the set WB.
The in-camera portrait profile sets the blues a bit purpleish, the brownish tones are
VERY different compared to RT default, and the reds are less prominent. This stays
for the in-camera default profile too with slightly different contrast and gamma.
The key points are:
- a per camera default profile seems to be the way to go,
- most of the "human workflow" for figuring out a curve could be implemented as a simple
algo,
- for handmade curves the two curve approach is more flexible and therefore more reliable,
- if anyone has a good, reliable workflow for D200 portraits, it'd be appreciated.
Ingo, are you interested in exchanging D200 profiles, maybe working together on a RT
D200 default profile?
Reported by gyurko.david@e-arc.hu
on 2014-01-12 16:22:13
Hi David, of course I'm interested in exchanging D200 profiles. I don't have a good
starting profile, because since about three years I take most of my photos with the
D700.
But as a have some thousand D200 NEF files, I would be interested in a good profile
and of course will help to improve your profile. So yes, let's start with a profile
from you (you know my mail-address).
Ingo
Reported by heckflosse@i-weyrich.de
on 2014-01-12 20:11:00
Summary:
- Bundle curves which emulate OOC-JPEG
- Store them in DCP profiles, or
- Store them in .rtc files so you can see and edit them, or
- Use whole PP3 files
Some cameras have no development modes so making a curve for them is easy, while other
models provide many modes and one would need to make a curve for each. I wouldn't be
surprised if there were many models for which replication of the OOC-JPEG in RT using
curves alone is impossible. If anything is to be done in this direction, it must be
compatible with all supported cameras; If using curves alone is not enough to replicate
OOC-JPEG for some cameras (D200@portrait?), then we should not take this approach and
instead ship PP3s.
Once an approach is chosen, we can use the forum to ask people to help out. Some requirements:
- Send in raw+JPEG bundles, the JPEG must come from the camera.
- They must document their camera's settings (development mode, contrast/sat/etc settings,
etc).
- They don't have to get the raw looking like the OOC-JPEG, they can leave it to us.
- The effect should be made using the minimum required set of RT tools. There should
be a certain order of tools used to get that look and once the look is achieved no
other tools should be touched, e.g. "Start with (Neutral), then use the Exposure slider,
then use Tone Curve 1/2" etc.
Reported by entertheyoni
on 2014-01-13 09:45:57
#63: I think we should at first focus on the default camera JPG settings (or alternatively
the "neutral" or "natural" setting), otherwise there are far too many picture styles
and combinations of settings (contrast, saturation) to keep track of. More often than
not, it won't be that difficult to achieve these different styles (e.g. by increasing/lowering
contrast or saturation/vibrance) once the default look is in the right ballpark.
I would keep it as simple as possible and just offer a starting point which gives the
same exposure and global contrast like a default camera JPG. I think this can be done
with a single tone curve per camera (ideally embedded in the DCP) and baseline exposure
per camera (as explained by Anders above). Users then have a starting point that looks
broadly similar to the embedded preview (same exposure and global contrast) and can
then use RT's many tools to further edit the image and add saturation, vibrance, contrast...
IMO trying to replicating the camera JPEG colors will be much too difficult as the
camera maker's JPEG engine will be using a different color profile than the one RT
uses and use different algorithms for adding saturation or even introduce a certain
color shift to get a "warmer" look that appeals to people. It could probably be done
with a combination of HSV equalizer and some of the new curve tools, but it will be
complicated and messy I guess.
Reported by stefan.ittner
on 2014-01-13 12:35:19
Isn't the simpler starting point for anyone who wants the "camera look" to use Adobe's
"camera" profiles together with the embedded in these dcps tone curve.
Torger added correct rendering for these dcps although looks like it works for the
relatively recent models (from about a year old and later), but not with older
http://rawtherapee.com/forum/viewtopic.php?f=3&t=5254&p=35570#p35570
Reported by iliasgiarimis
on 2014-01-13 22:19:29
#65: That would be nice, but I doubt that they alone give a camera-like rendering when
used in RT. At least those I tried looked not very similar to the camera-created JPEGs
(too little contrast and very subdued colors). I guess Adobe uses some additional processing
steps in LR to get them to look good.
Reported by stefan.ittner
on 2014-01-14 19:51:31
Stefan, I find these dcps give almost too much contrast when used with their embedded
tone curve ..
Reported by iliasgiarimis
on 2014-01-14 20:08:20
Oops, seems I have messed something up in my first test. I tried again and now the contrast
is indeed very similar to the camera JPG. Still, the colors are not good at all, but
maybe that's due to the bug you linked above (I tested for the Canon S110)?
Still, the problem is that there are many cameras for which there aren't any Adobe
*camera* standard profiles (e.g. all Pentax DSLRs more recent than the now ancient
K-7); and the regular "Adobe standard" DCPs apparently don't contain a tone curve.
And then there's the problem that Adobe won't allow us to bundle these profiles with
RT :) Of course users are free to download them separately and use them in RT, but
few people will bother to do so I guess...
Reported by stefan.ittner
on 2014-01-14 20:58:48
The current default profile with auto-levels has increasingly become my pet hate of
RT ;)
I very strongly think it's not a good idea to have auto-levels in the default profile,
and after 4.2 has been released and we start to work with 4.3 I'd really like to bring
this up again.
In my shooting style the auto-levels makes bad decisions more than half of the time.
I have snow in many images, I like shooting in soft light and many thus have a natural
low contrast etc, all those look plain wrong with auto-levels enabled.
What should be default, is what is default in any other raw converter and in camera
jpegs, ie a predictable "film like" behavior. If you expose an image one stop brighter
than another, voila it actually becomes one stop brighter when you look at it. If you
shoot in low contrast conditions, voila, the image actually looks low contrast when
you open it. It simply works as cameras have always worked, and I see no reason why
RT should be different from all the others.
I'm less concerned that it should actually exactly match how the camera renders jpegs.
Adobe Lightroom or Capture One does not match camera jpegs well either, they have their
own color and film curves. The key thing is that opening an image should give predictable
output.
For this some sort of "film curve" and a "baseline exposure" is needed.
There are numerous of ways to solve this. One is to do like Adobe Lightroom and embed
the curve and baseline exposure in the DCP, and when you open a file all contrast/curve-related
settings are neutral, except that the DCP color corrections, curve and baseline exposure
is applied. This is practical indeed, but probably not the right thing for RT, as RT
has the tradition of showing the curves to the user. Today the default is not to apply
the DCP curve if the DCP has one.
Having a separate baseline tool which includes a curve and baseline exposure is a related
idea, you could have options to get the curve from the DCP profile, choose it from
a number of presets, or make custom. One preset could be "analyze embedded thumbnail
and adapt to match", which could be kind of cool.
Another option is that instead of adding a new tool have different "auto-levels" modes,
where the new default one would be to set the curve and exposure slider only, possible
one can use the highlight compression and contrast sliders instead of setting the curve,
but then we stick to RGB processing, an advantage with the curve is that we can use
film curve or weighted curve instead.
It should be noted though that third party DCP profiles that have curves embedded,
like those coming from adobe, are designed with the curve in mind, ie it expects it
to be applied. A curve affects how color is rendered, so to get the look the DCP designer
had in mind one need to apply the same curve.
The difficult thing with this will not be implementation, but deciding how to introduce
this without messing too much with how RT currently works.
And of course agreeing that it's a good idea.
Note that I don't have anything in particular against auto-levels as a feature, there's
"auto-tone" in Lightroom which does the same thing (it's often bad results there too),
what I'm against is that it's default. If you always shoot landscape scenes with high
dynamic range in them auto-levels usually makes a good result, but there are other
shooting styles than that.
Reported by torger@ludd.ltu.se
on 2014-05-15 08:27:16
I agree with your point of view. I personally never use the default profile with auto
levels because of the same reasons: Far too unpredictable output and well-meant adjustments
that too often ruin the whole intention of a picture (e.g. low contrast scene). But
an almost bigger problem IMO is that its adjustments are only based on adding a symmetric,
standard RGB mode tone curve via the exposure contrast slider.
I like the idea of having a transparent baseline tool which includes baseline exposure
+ Adobe-style tone curve and where people can see what is applied and change it when
needed. Maybe we can offer several options: "Flat" (i.e. no curve), "standard" (normal
curve) and maybe a third one for high contrast scenes with a long highlight roll-off
similar to negative film.
Auto-levels could IMO be made more useful by acting *after* this baseline tool and
would already take into account the contrast & brightness changes from the base tone
curve.
Reported by stefan.ittner
on 2014-05-15 10:46:33
- Whether the default profile has Auto Levels enabled or not makes no difference to
me, since its so easy to make one's own default profile tailored to one's camera. I
don't know what our competent users want (by 'competent' I mean people who regularly
use RawTherapee, read RawPedia, understand the tools). I could run a poll in the forum
and on Google+. The Google+ poll I ran for the splash screen was very successful, many
voted.
- Auto Levels works as a great starting point on my shots where exposure tweaks are
needed. I mostly shoot high dynamic range landscapes and interiors for panoramas -
these I shoot manually and tweak manually. More often in the past but currently less
so, I shot portraits, events, concert shots, stuff like that. My camera has crap auto-exposure
calculation which is why I mostly shoot in manual mode, or seldom when I do use auto
mode then I have -1EV compensation to be safe, though sometimes it still clips highlights.
Either way, AL does a good job on the shots I test - not just mine, but the ones in
bug reports, PLAY RAW, etc.
- Few people shoot mist and snow, most shoot in crap midday light (based on what I
see in Google+ photo communities, Google Images, other social media, and so on). AL
works well there.
- Auto Levels is not intended to simply get the histogram to touch both ends but to
also get the mid-tones to a certain level. There could be a button to simply get the
histogram to touch both ends in the Exposure tool via sliders, or in the RGB and Lab
tone curves.
- I would not like any curves applied without my knowledge.
- Why don't people who want camera-like output make themselves a camera-like default
PP3?
Reported by entertheyoni
on 2014-05-19 11:23:58
Currently some DCPs have a curve and some don't, and so opening a photo results in:
There are several ways of handling this, I will list them in order of my preference:
One thing I am sure of is that our current "Default*" profiles need to change due their unpredictable results and not taking the DCP Tone Curve into account, and it would be nice to do this for 5.4.
@Beep6581 I can try working on integrating histogram matching, but realistically I'm not sure I can do it before 5.4 (unless you want to delay 5.4 even further...). Among the options above, I'd go for 2. If you want something more sophisticated, we could bundle profiles with "typical" tone curves for various camera brands (these could be generated by an enhanced version of my histogram matching script that takes the median of a series of shots for example), and then use dynamic profiles by default to activate the proper tone curve for the given camera maker (falling back to a generic one). This would also serve as an example of dynamic profile rules for new users... Or do you think this is just too complicated and not worth it?
@agriggio +1 for the histogram matching, but also for doing that for 5.5!
@agriggio as @heckflosse wrote - would be cool at some point to have this, but not necessarily for 5.4. If RT will have this feature in the future (even if it won't but especially if it will), then no point spending time on making median-average curves - we can't realistically do that for more than a few camera models anyway, I assume models from the same maker are not consistent so we would need to deal with "sets", and there are better ways of spending the time it would take...
I was thinking about generic camera brands, not individual models. And it wouldn't even need to be accurate, just in the ballpark. We could even use fake names to make this clear (say, "Nikonian", "Canonical", "Sonyc"... :stuck_out_tongue_winking_eye: ) I'm partially kidding, but I thought it could be feasible. Anyway, not a big deal, as I wrote option 2 would still be better than the current default IMHO.
I'm guessing here, but I don't think it's likely that my 12 year old Pentax K10D's curve would look like a Pentax K-1's curve, ditto for other brands. I looked at all of your histogram-matched PP3s from the forum and they weren't that different from each other, so I think given that we're already accepting variation between the OOC-JPEG and the suggested brand-median result, we can cut a lot of work out by taking it one step further and thinking of our DCP curve (which I'm suggesting to ship as an editable Exposure Tone Curve 1 instead) as a median of all the brand-median curves ;)
I'm not opposing the idea, I just don't think it's time well spent and not future-proof if RT is to have the histogram-matching feature some day.
Having said that, it occurred to me that, for curiosity's sake, we could run your python script directly on the RPU server which already has a collection of unique raw files per model (20GB, though ideally we would need many raw files per model). This is the folder structure: http://raw.pixls.us/data/ @agriggio if you could edit your script so that it can make those median curves by make, @andabata was willing to run it directly on the server for us.
let's go with the simple solution
Now that we have histogram matching, how about this for a default pp3?
[Version]
AppVersion=5.4
Version=330
[Exposure]
Auto=false
Clip=0.02
CurveMode=Perceptual
CurveMode2=Perceptual
Curve=0;
Curve2=0;
HistogramMatching=true
[Sharpening]
Enabled=true
Method=usm
Radius=0.5
Amount=200
Threshold=20;80;2000;1200;
OnlyEdges=false
EdgedetectionRadius=1.9
EdgeTolerance=1800
HalocontrolEnabled=false
HalocontrolAmount=85
DeconvRadius=0.75
DeconvAmount=75
DeconvDamping=20
DeconvIterations=30
[White Balance]
Setting=Camera
Equal=1
TemperatureBias=0
[Color Management]
InputProfile=(cameraICC)
ToneCurve=false
ApplyLookTable=false
ApplyBaselineExposureOffset=true
ApplyHueSatMap=true
BlendCMSMatrix=false
DCPIlluminant=0
WorkingProfile=ProPhoto
OutputProfile=RT_sRGB
OutputProfileIntent=Relative
OutputBPC=true
Gammafree=default
Freegamma=false
GammaValue=2.2200000000000002
GammaSlope=4.5
[LensProfile]
LcMode=lfauto
LCPFile=
UseDistortion=true
UseVignette=true
UseCA=false
[RAW]
CA=true
[RAW Bayer]
Method=amaze
[RAW X-Trans]
Method=3-pass (best)
@agriggio I was thinking of something very similar. I'm not sure what to do with the DCP look table - I think it should be enabled by default since we're using a tone curve in FIlm-Like mode by default. @atorger what do you think?
Apart from the DCP look table, what about re-enabling HLR via default profile? While it was almost always active in the past due to #4249, nowadays it's always disabled.
I propose to delete our current Default*
PP3s, to bundle all 6 of the following profiles, and to set either "Auto-Matched Curve - ISO Low
" or "Standard Film Curve - ISO Low
" as the default for raw files:
https://filebin.net/is4tqh4qcqrrmf3v
Thoughts, and stuff:
Which profile do we set as the default for raw photos?
While I personally prefer a linear curve, it's clear from user comments that they generally would like something like looks like their out-of-camera image. As such, I think we should set Auto-Matched Curve - ISO Low.pp3
as the default. We have a few weeks of testing until a final release - we can always revert to Standard Film Curve - ISO Low.pp3
if histogram matching turns out to be unreliable (it hasn't failed a single time in my experience yet).
I agree with everything. Just one note: I saw that the Standard Film Curve
profiles have highlight compression set to 90
, is that intentional?
Yes, it prevents ugly highlight-clipping transition zone discoloration. Have you found it to be a problem?
Have you found it to be a problem?
I haven't tested this profile specifically, but in my normal use of RT I have never found a "one size fits all" value for highlight compression -- it seems to me that this is very image-specific. But maybe I'm wrong.
I have never found a "one size fits all" value
Agreed. With the changes in the last year, HLC is less aggressive and seems to do no harm at values <100. The default profile is nothing more than a way for a new RT user to get a decent-looking starting point. If you don't mind, I would keep HLC=90 for now - we can always remove it before the release. If it does more harm than good - please report!
P.S. in c2e7d924a171a36da68ac5ca482c0d52e5a60641 the commit message says,
False color suppression steps set to 2 - this affects both X-Trans and Bayer raw files.
I made a mistake - the CcSteps
I changed affects only the X-Trans section, but the Bayer section already had that key set to 2
, so both are 2
.
I processed concert photos using the new PP3s. I'm generally very happy with them.
--edit I split off what I wrote here into a new issue #4341
Originally reported on Google Code with ID 2071
Reported by
torger@ludd.ltu.se
on 2013-11-20 19:25:54