filmicrgb defaults produce unusably contrasty images

[bobobo1618]

Describe the bug

I'd integrated filmic into my normal workflow, which went something like this:

• Load photo
• Turn off base curve
• Adjust exposure to roughly centre the histogram
• Turn on filmic
• See something that looked roughly usable straight away
• Fine-tune exposure
• Play with middle grey, white and black sliders until I had something I liked

With filmicrgb, at least the version I'm running currently (0eb00ef), the defaults produce super contrasty images that I consider unusable. I'm yet to find a case of an image that looks usable with filmicrgb's defaults.

I'm also not sure what I have to adjust to get usable images out of it again. In particular, I can't for the life of me figure out how to let the shadows keep a little bit of colour instead of going straight to black (compare the filmic and filmicrgb screenshots below and look at the water).

Screenshots

Here's an example progression of my workflow, with both filmic and filmicrgb:

• 1 - original image
• 2 - no base curve
• 3 - exposure adjusted
• 4 - filmic with default settings
• 4 - filmicrgb with default settings

Platform (please complete the following information):

• OS: Arch Linux
• Version 2.7.0

[Nilvus]

filmic rgb replace filmic, so both shouldn't be used together. If you use, as I understand your post, together on same image, it's normal that you have this issue.

Good use is to use base curve OR filmic OR filmic RGB on an image.

Your post is not clear and could also be understandable that you replace filmic and filmic RGB. In that way, on background they are the same but change to one to another needs some adjustments to have same rendering. A suggest you to view (quite long but useful) Aurelien Pierre last videos on Youtube. That will help you on your issue (that it's not a darktable issue but just how to use the module). So think also on closing this issue (you could also use pixl.us forum to discuss how to use it (after viewing the video if possible). Thanks !

[bobobo1618]

I'm not using them at the same time. I understand that only one of filmic, filmicrgb and base curve should be used at the same time, that isn't the issue and the screenshots I added of filmic and filmicrgb only use one of the two (and base curve is turned off for both).

I'm happy to watch the video but I don't know which in the series is relevant and would appreciate a more precise reference.

Regardless of how to use the filmicrgb module, I think that the defaults producing unusable images is a regression relative to filmic and worthy of a bug.

[Nilvus]

Searching Aurelien Pierre is not difficult to do in Youtube. I'm sure how to do that. And you will see his last videos.

Default settings are a quite difficult thing to do and none default settings could suit all images. Sometimes filmic ones are better, sometimes filmic rgb ones are the best ones. So, as I use it, I don't find it's a bug. Anyway, yes default settings, depending on your images could be worse. On what I see on your images, they're not so bad and just need a little tweak.

Think also that default settings are just... default settings. They're proposals only, just saved defaults presets. And each photograph has it's own visual preferences, so all default settings have to be tweaked depending on what you like. That why new presets could be created.

[bobobo1618]

Default settings are a quite difficult thing to do and none default settings could suit all images. Sometimes filmic ones are better, sometimes filmic rgb ones are the best ones.

After I un-deprecated filmic and recompiled, I went through a few hundred photos I'd processed with filmicrgb. Every single one, without fail, looked better with filmic's default settings than with anything I was able to get out of filmicrgb (including its default settings). That's wildlife, landscape, portraits, street, daylight, overcast, night-time, everything.

This isn't a "sometimes one is better than the other problem", this is a "filmicrgb's defaults are a regression" problem. Just look at my filmic vs. filmicrgb screenshot examples from earlier. That's what I see consistently.

It's likely that I'm not able to get anything usable out of filmicrgb because I don't know how to use it yet but that's a separate issue to the default settings being a regression.

[bobobo1618]

I'll add that the reason this matters to me is that this isn't a new shiny alternative to filmic, it replaces filmic. My workflow, which could be summed up as "turn base curve off, turn filmic on" and usally took ~2s per photo is now broken and there are no fixes or workarounds.

Given that it appears filmicrgb is going to require extra work for each photo, my preference would be to un-deprecate filmic and let the two co-exist for now.

Also, I'm always in the IRC channel on Freenode if anyone wants to discuss there. I'm in Europe and awake during normal European hours.

[Nilvus]

So, using darktable 2.6 seems to simply be more suitable for you if default settings of filmic were ok for you.

If I understand well, you want a just one click by module setting that works on all images. That way, darktable is probably not the better tool for that. darktable is an advanced processing raw image, not a one click tool. Some other software are better for that. darktable is for photographers who want to have the most control possible on their images. Personally, I never use default settings (except the first time to understand the module behaviour and prefer my default settings (to have my way I see photos).

And you could maintain your workflow here by created a new preset (after tweaking an image with the filmic rgb settings (I don't talk about defaults presets here)). Or, you could just create a style from an already edited image with filmic from darktable 2.6.x and copy that style on new images (or even made the filmic module automatically applied on new images. I didn't test that but as the module remains on darktable, it normally will be applied these ways on the new images.

That will be my last reply here. I can understand that you don't like that but filmic will not be un-deprecate because filmic rgb replaces it and I see many users find this new version easier to use. Again, it just need a little time to change habits from previous version. Not just a one click. That a change yes, not a bug ! Sometimes, a new version could change a workflow, especially on an advanced software like darktable.

[bobobo1618]

If I understand well, you want a just one click by module setting that works on all images.

No, I don't want some kind of magic. I rarely just turn filmic on and walk away, I also make small adjustments.

What I want is for the default settings of filmicrgb to give me a usable starting point. The image shouldn't look broken as soon as I flick it on.

darktable is an advanced processing raw image, not a one click tool.

Sure, darktable is advanced software that gives you a ton of flexibility. That doesn't mean it makes sense to accept regressions.

Or, you could just create a style from an already edited image with filmic from darktable 2.6.x and copy that style on new images (or even made the filmic module automatically applied on new images.

That doesn't allow me to make adjustments to filmic settings.

filmic will not be un-deprecate because filmic rgb replaces it

filmic rgb is intended to replace it but part of the issue I'm getting at on this bug is that it's not a replacement so much as an alternative. They don't do the same thing right now so one isn't really a replacement for the other.

That a change yes, not a bug

My problem isn't that things are changing, it's that things are regressing. Defaults used to look reasonable. Now they don't. Something that worked now doesn't. That's a regression.

[bobobo1618]

Also, @aurelienpierre, so you're aware of this.

I want to be clear that I'm not opposed to filmicrgb, it's just that right now, for me at least, it's not a drop-in replacement for filmic.

To give a concrete idea of what I want, I'd like one of these things to happen (in order of preference):

• filmicrgb's default settings should give a useable starting point (i.e. look similar to filmic's default settings or base curve's default settings).
• Add a preset or add documentation on how to configure filmicrgb to work like filmic in such a way that it can be applied en-masse (i.e. I can set the preset as the default setting).
• Un-deprecate filmic so that people have time to learn and get used to filmicrgb before switching. filmic could be removed later.

[aurelienpierre]

Hi, 3h20 of documentation : https://www.youtube.com/watch?v=3FFU-Ltvm0I&list=PL4EYo8VotTsiZLr3BqGeBRj-qYGO63bIv

filmicrgb's default settings should give a useable starting point (i.e. look similar to filmic's default settings or base curve's default settings).

I agree. Now, how do I do that, taking into account all the possible changes that can have happened before in the pipe ? I tried, I failed.

Add a preset or add documentation on how to configure filmicrgb to work like filmic in such a way that it can be applied en-masse (i.e. I can set the preset as the default setting).

Video in on the way, preset is impossible.

[junkyardsparkle]

Just look at my filmic vs. filmicrgb screenshot examples from earlier. That's what I see consistently.

The upper-right corner of your filmicrgb screenshot looks suspicious to me. Look at the weird halo between the branches and the weirdly dark sky... you have more going on here than just filmic RGB, it doesn't do things like that by itself (it's a global, not local, tone mapping).

Adjust exposure to roughly centre the histogram

Note that if you're doing this with badly clipped images, it will create problems which filmic won't solve for you.

[junkyardsparkle]

As a counterpoint, here's my attempt at an "apples-to-apples" comparison. Image is a failed attempt to capture a fleeting rainbow with a small-sensor compact (the RAW files it produces are certainly nothing special in terms of dynamic range).

First, flat camera ouput with only the auto-applied CLUT profile:

This is the result of enabling old-filmic:

This is the result of enabling filmic RGB:

This is the result of simply clicking the auto-adjust in old-filmic:

This is the result of simply clicking the auto-adjust in filmic RGB:

They're different, but nothing "unusably contrasty" going on there. :-)

[sboukortt]

I sometimes get unusably contrasty images with auto-adjust, but that’s because for some reason, filmic RGB detects black as being at -1 or -2 EV.

Here is an example with no base curve, no filmic, just lens correction and exposure:

Filmic RGB, default settings:

Filmic RGB, “auto-adjusted”:

Here are the settings that it detected:

I have seen it happen with images from both of my cameras, maybe more often with this one but I am not completely sure.

In contrast (no pun intended), the old filmic had a tendency to detect it at the other extreme, at -16 EV. I think it was less jarring to look at.

[aurelienpierre]

Please note that filmic comes late in the pipe and is not aware of the previous steps of adjustments, which can be pretty much anything.

The success of the auto-tuners depends on the image corrections that have been performed before filmic. Open the "enabled modules" tabs to see the order of application of modules, anything below filmic in the list comes before in the pipe.

RGB values equal to zero or negative at the input of filmic will make the black auto detection fail every time. Also filmic has no way to make the distinction between legitimate black and random noise. So watch out the raw black level and black level correction in exposure module.

If default settings don't please you, change them and create your own presets.

If auto-tuners fail, set the parameters manually. The assumptions under which they work are detailed in the user manual, if these conditions are not met, simply don't use the auto-tuners.

I'm not willing to discuss that topic any further. That module was designed to give users control, yet it seems all they want is a two-click magic and one-size-fits-all default parameters.

In contrast (no pun intended), the old filmic had a tendency to detect it at the other extreme, at -16 EV. I think it was less jarring to look at.

The old and the new filmic default to -16EV when black is invalid. Having anything but -16EV means a valid black has been found. However, in that graph, your latitude is too large and you are clipping shadows with that curve.

[sboukortt]

Far be it from me to be annoying or ungrateful for your great work, but if you don’t mind, I would like to understand a bit more what is going on:

Please note that filmic comes late in the pipe and is not aware of the previous steps of adjustments, which can be pretty much anything.

The success of the auto-tuners depends on the image corrections that have been performed before filmic. Open the "enabled modules" tabs to see the order of application of modules, anything below filmic in the list comes before in the pipe.

In this case, it was just lens correction and exposure, and it happens even if I disable them. Does that mean that it has to do with the RAW file itself?

If auto-tuners fail, set the parameters manually. The assumptions under which they work are detailed in the user manual, if these conditions are not met, simply don't use the auto-tuners.

(If you happen to know,) where can the current manual be found without having to build it from source? The website only has the 2.6 manual, using darktable’s help tool on filmic RGB shows “there is no help available for this element”, and the Windows build doesn’t appear to include it either (it just has an empty directory for it):

$ ls /c/Program\ Files/darktable/share/doc/darktable/usermanual/
$

The old and the new filmic default to -16EV when black is invalid. Having anything but -16EV means a valid black has been found. However, in that graph, your latitude is too large and you are clipping shadows with that curve.

It also clips them with contrast, latitude (and shadow/highlight balance) all set to minimum. Roughly how is the black level estimated? At least on this particular image, it seems to detect it at -1 EV pretty much independently of the raw black point (even 0).

That module was designed to give users control, yet it seems all they want is a two-click magic and one-size-fits-all default parameters.

I don’t think this is an either/or question, a good starting point can be useful even in a manual process with a lot of control. I certainly liked that in the previous version of filmic. (Obligatory xkcd.) As another example, I think I might like it if my camera had a “semi-manual” mode where the press of a button would set the exposure settings to what they would be in automatic mode and then the camera would let me go from there and adjust them as usual with the dials.

[sboukortt]

Looking at the source of the manual, I seem to have found the relevant bit:

https://github.com/darktable-org/darktable/blob/f788bfff36864f69a1d2ba8b86d20e8e4fde9ef1/doc/usermanual/darkroom/modules/tone/filmicrgb.xml#L105-L107

After denoising, the black point is still quite aggressively high but it does seem a little better. Is it estimated in terms of percentile or something like that?

Edit: another relevant bit! Turns out that Aurélien thought of a lot in advance :)

https://github.com/darktable-org/darktable/blob/f788bfff36864f69a1d2ba8b86d20e8e4fde9ef1/doc/usermanual/darkroom/modules/tone/filmicrgb.xml#L92-L96

Thanks for writing the doc (and the module), and sorry that it gets so little attention.

[atomicwrites]

Oh wow, thanks @sboukortt. I'd been going crazy looking for the documentation. It isn't anywhere in the site.