IOP Modules: Which Group?

[elstoc]

In testing a recent PR I ran the script tools/iop-layout.sh to change the module group display order. However, I noticed that quite a few of the modules seemed to move to different groups after I ran the script, so there is a discrepancy between the shell script and the default groups that modules are assigned to in darktable 3.1.

Here's a list of the mismatches. I've highlighted in bold what I think might be the 'correct' grouping (though a few of these are very subjective). Where I think both might be incorrect I've put what I think is correct in brackets. In addition there's one (color transfer) that I can't find in darktable at all and two that aren't mentioned in the shell script (negadoctor/tonequal).

TBH I'm not all that bothered really which groups they should be in (we could just adjust the script to match current darktable gui grouping) but it would be nice to be consistent.

Module	Tools Group	GUI Group
graduatednd	Basic	Effect
colorin	Basic	Color
lens	Basic	Correct
colorout	Basic	Color
ashift (perspective corr)	Basic	Correct
rotatepixels	Basic	Correct
scalepixels	Basic	Correct
tonemap	Basic	Tone
profile_gamma (Unbreak Input Profile)	Basic	Color
filmic	Basic	Tone
filmicrgb	Basic	Tone
bloom	Tone	Effect
colisa (Contrast/Brightness/Saturation)	Tone	Basic
atrous (Contrast Equalizer)	Tone	Correct
shadhi (Shadows Highlights)	Tone	Basic
colormapping	Color	Effect
colortransfer	Color	???
colorize	Color	Effect
lowlight	Color	Effect
splittoning	Color	Effect
colorreconstruct	Correct	Basic
highlights (recovery)	Correct	Basic
liquify	Effect	Correct
retouch	Effect	Correct
sharpen	Effect	Correct
spots	Effect	Correct
negadoctor	?	Basic
tonequal	?	Basic (Tone)

Thoughts?

[TurboGit]

BTW, we probably want to add new modules like negadoctor and filmicrgb into the iop_layout*.py scripts.

[johnny-bit]

Hi! I've checked the list, thought about it and I recommend following:

module	desired group	comment
graduatednd	Effect
colorin	Basic	it's required there imo
lens	Correct	it's correction :)
colorout	Basic	same as colorin
ashift (perspective corr)	Correct	still - correction
rotatepixels	Correct
scalepixels	Correct
tonemap	Tone
profile_gamma (Unbreak Input Profile)	Basic	needs to work in line with colorin imo
filmic	Basic
filmicrgb	Basic	filmicrgb (and v4) is like supercharged everything to use instead of basecurve. need to check with @aurelienpierre
bloom	Effect
colisa (Contrast/Brightness/Saturation)	Tone	should be kicked out from basic. (only saturation doesn't fit into tone group)
atrous (Contrast Equalizer)	Correct	it's very powerfull correction tool imo, doesn't really fit into tone modifiers
shadhi (Shadows Highlights)	Tone
colormapping	Effect	it's effect that maps colors, not color adjustment like the tools in color group.
colortransfer	effect	it's deprecated
colorize	Effect
lowlight	Effect
splittoning	Effect
colorreconstruct	Correct	Technically should imo be in corrections
highlights (recovery)	Correct
liquify	Correct	it IS correction ;)
retouch	Correct
sharpen	Correct
spots	Correct
negadoctor	Basic	should be in basic IMO (prefferably instead of invert and kick invert to color group)
tonequal	Tone	should be in tone instead of basic (basically filmic and tone equalizer should switch paces imo :P). need to check with @aurelienpierre

I wonder if some other modules would need reshufling ;)

also if there is gui change needed, documentation should follow.

[aurelienpierre]

I think it's time for a redesign of groups, it's just a massive clusterfuck right now. To be honest, I don't even use the group tabs anymore, I only ever lookup modules by their name in the search bar.

A long time ago, on the mailing list, I advocated that we should clearly separate what belongs to technical editing (generally happening early in the pipe and bounded to hardware constraints) from artistic editing (generally happening late in the pipe, and completely free from technical constraints).

As you see, "correct" has a different meaning depending on people. "basic" is… well, entirely subjective. "tones" and "colors" are very difficult to separate, given they are linked in human vision as well as in most algorithms and modules… Then, there is the never-ending discussion about whether modules should be grouped by use frequency or scope/meaning.

So, the way I see groups, is:

1. technics: everything related to objective technical matters (sensor, lenses, and color spaces) such as profiling, gamut and tone mapping, damaged parts reconstruction, perspective and lens correction, etc.
2. grading: everything related to subjective primary (corrective) and secondary (creative) color and tone corrections,
3. (special) effects: retouch, liquify, bloom, sharpness and such.
4. most used modules: to maybe keep lazy people happy and avoid them to browse for exposure and white balance.
5. favorite (which, to me, is redundant with the previous one).

Then, we have the weird kids like contrast equalizer and local contrast. What they strictly do is massage contrast locally at various scales. What they produce is tone mapping or sharpening or blur, depending on how they are set and used. So, they are better in effects.

There are only 2 things I'm sure of:

1. basic sucks. Is it basic feature-wise (as in "you will need to use that all the time"), or step-wise (as in "fundamental processing steps") ? Nobody knows.
2. color vs. tones separation is nonsensical. The proof is most modules in there affect both.

[elstoc]

favorite (which, to me, is redundant with the previous one)

Call it a custom list instead, perhaps. I suppose people might want to include modules in the 'favourites' list that they don't use very often - there should at least be one group that users can customise without having to change config and 'favourites' is as good a name as any. Apart from that I agree with everything you say.

[parafin]

How do define most used? Do you mean keep statistics of actual user use or what?

Also I guess there will be an argument that sharpness belongs to technics, since it's auto-enabled (or at least was for a long time).

I personally never used groups except for favourite, because they didn't make much sense to me (I couldn't guess where to look for a module). Your suggestion sounds much more logical than current system.

[aurelienpierre]

Also I guess there will be an argument that sharpness belongs to technics, since it's auto-enabled (or at least was for a long time).

Sharpness does not belong to technics since it's not proper lens deblurring (with proper point spread function as a thin lens model), but hacky acutance enhancement. Lens blur deconvolution would go there, but that doesn't exist in dt (yet :smiley:).

How do define most used? Do you mean keep statistics of actual user use or what?

I suppose it's the same issue as the "basic" group indeed.

[aurelienpierre]

there should at least be one group that users can customise without having to change config and 'favourites' is as good a name as any.

Agreed. I meant it's redundant to have both a "most-used" and a "favourite" group, but of course the user-customizable one should be favoured.

[johnny-bit]

How do define most used? Do you mean keep statistics of actual user use or what?

we have db with module usage history :) no problem in querying it ;)

[favourite being redundant] Call it a custom list instead, perhaps.

Or module bookmarks? dunno. I use favourites similarly to @parafin, some modules are in groups I wouldn't expect (as evident by my reply to @elstoc ) and so I use that to have less searching in case I forgot module name.

So, the way I see groups, is: [technics, grading, special effects, most used, favourites]

That's... sensible. Additionally with making #3648 a bit more of "favourite adjustments" (as I mentioned in #5063) favourites tabs would be quite useful.

[elstoc]

We could just go all the way and let the user create their own groups (in preferences, without having to run shell scripts).

[johnny-bit]

been there, done that (in completelly unrelated app, we were allowing users to define their own toolbars). Let's just say that 99% of users never adjusted their toolbars and of 1% that did there was non-zero amount of "i don't know what it did and now it's bad" ;)

but my "be able to configure everything" approach is yellin "hell yes".

[aurelienpierre]

We could just go all the way and let the user create their own groups (in preferences, without having to run shell scripts).

You still need a sane default though. And careful about adding more preferences… it is the Autodesk way: it backfires after a couple of years, when you have 48 pages of GUI settings and you need 250 pages of docs to explain how to set them.

But there is still the option to let users drag and drop modules to tabs, in GUI, though.

[elstoc]

Agree we'd still have to argue over the default groups but at least being able to easily move modules between groups would mean we don't have to worry too much about agreeing, initially, what module goes where. Drag and drop is a good idea. Can I also mention #5129 and #3085. We need to better handle saving of the expanded/collapsed/scroll states of the groups so it's more consistent with the preference settings.

[codingdave]

I also never find my module in the group and use the search for everything. I vote for sorting according to their use frequency with a Hysterese and the search bar.

[elstoc]

sorting according to their use frequency

Sorting is based on the order of the module in the pixelpipe.

[TurboGit]

Moving for 3.4, feature freeze in 10 days for 3.2.

[aurelienpierre]

Well, @TurboGit, the PR is ready :yum:

[github-actions[bot]]

This issue did not get any activity in the past 30 days and will be closed in 365 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.