Multiple control points for "locallab branch"

[Desmis]

Hello all After 2 mounth whithout RawtherapeeI, I updated "locallab" branch with a modification allowing several "Control points". I put 5 control points, but it's easy to add more or less. The proposed system is far from perfect, there are still many issues to be resolved. But despite all the bugs, I prefer to distribute this change: 1) for opinions of principle ... is that we continue to do this way or we abandon ? 2) to get help from other developers, especially to handle events, and others optimizations or problems.

Now if you are very tolerant, and if you follow a rigorous process, it is possible to control 5 Points, for "Color and light", "Blur and Noise" and "Retinex"..in 2 modes : Preview and TIF/JPG

However I was not able to properly run the events, and I installed a work around.

To select control point, I added a slider with 5 positions, but due to my bad events management, I was forced to add another control that I called "Help to move control point". When you want to add (or move) a control point, you must manually change "help to move control point" (0 to 1, or 1 to 0... a check point does not work), just after change control point.

It works, but with bugs...I have tried many solutions for events :

• in improccoordinator.cc just near call to "aloListener->localChanged(dataspot, sp);" (line 903) (todo |= M_LUMACURVE)
• in locallab.cc, localcomputed_ with adjusterChanged() and others changes, but all is bad..sometimes it seems to work...but no!

Sometimes the system crash...

Monitoring "control points" is provided by a file * .mip (text file). eg for a file _ASC4029.NEF, a file _ASC4029.NEF.mip is create near the NEF file. (to prevent crash (??) or bad results, I always delete manualy "mip" file...and select "neutral" (profiles), for other testings!!)

for your advice and help.

Thank you

Jacques

[heckflosse]

@Desmis

Hello Jacques,

I just compiled your branch but can not find the new controls. Can you please post a screenshot?

[Desmis]

@heckflosse

Hello Ingo

Here a screenshot. http://jacques.desmis.perso.neuf.fr/RT/multi.jpg

You can see just above "shape", the 2 sliders "Control points and "Help to Move Control point"

Jacques

[heckflosse]

@Desmis

Hello Jacques,

I must have been completely blind or confused. Now, after building again, I see it. Thank you!

Ingo

[Desmis]

@heckflosse

Hello Ingo I think, I have found a way to solve (perhaps partially) "events"...Can I push a change, or wait ? Thank you

Jacques

[heckflosse]

Push :)

[Desmis]

@heckflosse

Ingo and all

I think I have partially solved the problem of managing events (I just push a change to locallab). Nevertheless not everything works properly, including "inverse" function.

Of course there are still issues to resolve, such as the optimization, but now I think we should be able to test properly :)

[sguyader]

@Desmis I just tried your latest changes and it looks very promising. In my quick test I was able to work on 3 separate areas, and it is easy to jump from one to another with the slider. I only noticed that when using "noise" in one area, modifying things in another area was changing slightly the preview of the "noise" area (I guess it makes RT recalculate the random component of the noise distribution, which in fact is not really a problem). Great job Jacques!

[Desmis]

@sguyader Sébastien

Thank you for this quick test :)

[Desmis]

I add controls for events :

• for inverse (color light, blurr, retinex);
• for the 2 methods : cursor and retinex

I hope no error...to test :)

if we admit that it works correctly, you can ask the following questions: 1) is there enough control points (actually 5). It is easy to fix another value, 3, 6, 10,...or make configurable via option ? 2) with the same general process, then there must be other types of control points : noise, sharpening, spot-removal, ...? (for this last point "spot removal", it needs a file with spot to elaborate algorithm)

Jacques

[Floessie]

@Desmis Bonjour Jacques,

is there enough control points (actually 5). It is easy to fix another value, 3, 6, 10,...or make configurable via option ?

I didn't have a look at the code let alone the whole feature, so please bear with me if this is a daft question: Why does the number of control points have to be fixed? I could image -- from a user's perspective -- that adding a CP when you need one would be nice. Sorry, if that idea is stupid for some obvious reason...

Best Flössie

[Desmis]

@Floessie Excuse my very bad english :) In fact, in an ideal world, it would be that the user does not have to make that choice.

In this case, I was led to make choices based on a hypothetical algorithm without being sure that will go run. Now it's run...

Probably we should be able, from the basic algorithm I imagined, find other solutions. I am open to any concrete proposal :)

[Floessie]

@Desmis

Excuse my very bad english :)

I don't have to: You write quite understandably. :)

Probably we should be able, from the basic algorithm I imagined, find other solutions. I am open to any concrete proposal

From the code I see, that CP 0 is special, as it is the base for the other 4 CPs. Apart from that, they seem equal.

Could it be, that the main problem for not having a dynamic number of CPs is the way they are passed between functions as int**? If so, storing them in a map of maps (e.g. std::map<Item, std::map<Subitem, int>>) could help (and would also make it more readable). Best would be a class holding CPs and a class modelling those CPs. But again, I'm not really familiar with the code. So, sorry if this is rubbish.

Best Flössie

[Desmis]

@Floessie

Hello My english is bad, but also I am a bad computer!

For me no problem, if you think, it is better to write the code in another way : dynamic CP, mutex, etc. You can change without problem :)

thanks in advance!

Jacques

[Floessie]

@Desmis

My english is bad, but also I am a bad computer!

:joy: Biggest understatement of the year. I mean: Who is the innovator here?

You can change without problem

I will, as time permits. Where can I find the names or meanings of the control points' attributes? (Edit: It's in the params.locallab.) I could at least help you with the storage classes and the mutex problem, but no so much with the GUI.

HTH Flössie

[Desmis]

@Floessie

I'll try tomorrow, to explain how control points'attributes works :)

Thank you again Jacques

[Desmis]

@Floessie I see your edit... I describe summarily the principle :

Principle of the algorithm

I don't use layers, or layers mask... Each control point has is own controls : transition, scope, etc. I think we can easily use the same principle for others controls (noise, spot removal, etc.) I think also we can combined algorithms from several control points...but not now :)

2 files are principally used :

• improccoordinator.cc and locallab.cc

2 files are also used but read only and directly derived from improccoordinator.cc

• dcrop.cc and simpleprocess.cc

Iplocallab.cc is slightly modified, I replace the luminosity curve, by a calculation

2 parameters are not differenciated and are the same for all Control points:

• transmission gain : 1) I don't know how to save parameters in a file; 2) I think it is not important because, Transmission gain is often a caracteristic of an entire image
• avoid color shift

In improccoordinator.cc and locallab.cc Creation of a mip file, near the file to be treated If this file does not exist, create with defaut parameters Creation of an array dataspot with 2 dimensions : 26, number of spot + 1: Initialize dataspot[x][0] with params.locallab values Read the mip file and initialize dataspot[x][sp] with new values and for each spot (sp)(except 0)

For each spot sp, except the current spot:

• attribute params.locallab with dataspot[x][sp]
• call lab_local

For current spot :

• call "aloListener->localChanged(dataspot, sp)", which call some functions in locallab.cc (GUI). It is a process create some times ago by Hombre (thanks to him)
• this function update preview GUI

• I searched a long time how to operate with various tests and finaly, I add in Locallab::localComputed_ (), a call as :

  listener->panelChanged (Evlocallabanbspot, anbspot->getTextValue());

  etc. *I think perhaps we can simplify ?? this call ?? Then, attribute to dataspot[x][sp], params.locallab, and dataspot[x][0]

• call local_lab

Then save the mip file.

In locallab.cc in addition to what I just described, I add a slider to change and select the current "control spot" We can certainly change that, for a dynamic system

I think we must optimized the code, I repeat, I am a scientist and an average computer...some functions are chinese for me

Jacques

[Floessie]

@Desmis I've just started to fix the "fake" mutexes. But I have one question: What are they supposed to protect against? Especially the one here. Should that be the same as this one?

Are they necessary at all in the light of cropMutex and mProcessing?

Best Flössie

[Desmis]

@Floessie

The current code is the result of numerous trial and error ... and I'm not sure that everything is useful because I brought in the development of many changes. That is why I asked the help of another developer to do a code review for its optimization. In the case of "Mutex" I think I have come to add to manage files * .mip because during the development I had malfunctions when reading and writing, but can be is it more useful?

Also do you are working on the dynamic management of the number of spots (essentially a GUI issue) or do you prefer that I examine the feasibility? jacques

[Floessie]

@Desmis I see. So the (single) mutex must protect the file accesses. No problem.

Also do you are working on the dynamic management of the number of spots (essentially a GUI issue) or do you prefer that I examine the feasibility?

Well, I've just started, but my plan is to support you with a class managing the *.mip files and a class representing the control points (which can then be passed around in a std::vector<> with dynamic size). I'll leave the GUI up to you. Sorry. :)

[Desmis]

@Floessie

Thank you :) I'll work on GUI after your modifications.

Jacques

[sguyader]

@Desmis Jacques, some RT users would love to have a new Local Lab binary. The link to the one I posted in june has expired. Would you agree if I make a new binary for Windows 64 bits, and post it as an experimental build, on the RT forum?

[Desmis]

Sébastien No problem for me. But it is an very experimental version. Thank you :) Jacques

[Desmis]

Sébastien No problem for me. But it is an very experimental version. Thank you :) Jacques

[Hombre57]

Hello Jacques,

I'm following this issue and branch from the back seat, seeing the progress and reactions from the other devs and users.

I still think that we should create a mechanism to enable general purpose local editing, and that's the reason why I returned. The soft-proofing branch and color picker branch has interfered in my agenda and will of doing that mechanism, and IMHO creating it before making the Gtk3 branch the main one would be prematurate, since it will be very invasive regarding the GUI.

Regarding the Spot removal feature, a stalled branch already exist for that and I need someone for the algorithm itself. I know that I could copy/paste it from other FOSS project, but I'm quite bad at deciphering a project's code and isolating the useful part. I could revive this branch (the GUI is mostly done) if someone is keen enough to help me on this.

So since you ask us about your branch, I'd suggest to put your branch aside and unit our force on this Big feature (again, let's make Gtk3 the new master first).

Cordialement,

---

Jean-Christophe

[Hombre57]

Btw, making a very invasive patch like Local Editing would require some kind of "mutex" developing mode, i.e. it will touch all the part of the code and other opened patch or branches would suffer from that. I'd recommend to put master completely on hold before the LocalRT can be merged to master. That's a collective decision and I wouldn't start this feature if I had to keep synchronized with an ever changing master and strongly changing LocalRT branch. I think Jacques knows what I mean.

[Desmis]

Hombre

No problem to help you for "Spot removal tool algorithm" according to my skills :)

jacques

[Floessie]

I've only invested 1.5h into the dynamic CPs, so no problem, if we don't pursue this branch as it is, but unite to get the feature mainline.

[Hombre57]

I'll merge master into the spot-removal branch and we'll try to finish it first then, and continue discussion in issue #2239.

[Desmis]

@Floessie 1) I would like to merge locallab with master 2) and more I will bring an improvement in the shape detection algorithm for each control point. This change only concern iplocallab.cc (and improcfun.h)

Can I push this change or must I wait ? Thank you Jacques

[Floessie]

@Desmis

1) I would like to merge locallab with master

I was referring to Hombres suggestion to develop a general purpose local editing and ditch this branch entirely.

Can I push this change or must I wait ?

Absolutely! Don't wait for me with anything. I can't contribute that often, and I have to help Ingo with some cleanups first before the RT5 release. Even if there are conflicts, I'm able to merge.

Best Flössie

[Desmis]

Done :)

[Desmis]

I remember that to achieve multiple controls points (for luma, contrast, chroma, structure,...), there are several problems.

What emerges from the GUI with several types of solutions: a) as currently implemented, which may seem a bit "primary", not optimized computing terms, but a priori that works b) alternative(s) as proposed Hombre, which will probably be more intuitive, and more efficient in computing terms...

Algorithms to control local editings: 1) with layers , masks, etc. (as Photoshop...) 2) without layers (at least for the user), as U-point (control points), as Nikon CNX2 or Nik Software The solution I propose (with Ingo optimization), is clearly 2) and can be used whatever the GUI :) Jacques

[michaelezra]

For best usability of local adjustments, it is best to try to design them in such a way that they could be applied in numerous ways:

1. multiple (unlimited) instances of adjustments should be possible
2. as a brush
3. as a gradient - linear and ellipse
4. as U-point

2, 3, 4 don't need to be implemented all at once, but it is worth designing this so that all methods are achievable. I use local adjustments with camera raw (extremely) heavily within Photoshop. On average, about 10-30 per image. ACR's implementation of this feature is excellent, also with the range of tools that are part of each adjustment. This is a worthwhile reference to keep in mind:) Sorry that I've been out of the game for so long, hope this is useful.

[Hombre57]

That's not the right issue to talk about it, but here is what I have in mind for local editing :

1. Add a double arrow button like the one of the Gradient tool to selected Slider adjustment to add on preview local editing. The slider wold still be effective and used as baseline value, then the local editing objects would modulate the value locally. That way, we should have an heterogeneous value and we'd still be compatible with the Batch editor. However the local editing wouldn't be available here, except for resetting the local edits. This is the easier part of the job.
2. On a second scheme, we could enable multi-profile per pp3, which opens a lot of possibilities ! Add to this possibility alpha channel support and fusion operator, and we're not far away of layer editing. It would let us do real HDR fusion, à la HDRMerge, or handle dual sensibility Fuji sensor, or stitch several images to create a panorama, or fusion several images to do focus stacking, etc.
3. Add bitmap support (i.e. painting) alongside the vectorial objects for local editing, but then the pp3 will have to handle binary objects, and that's not a simple affair for me.

My goal is doing point 1 first (and perhaps only, no promise for anything), then take some rest...

@michaelezra Instead of multiple instance of a tool (if I understood your point 1 correctly), I propose to have a single instance to keep the pipeline fixed as it is now, but modulate the value with unlimited amount of local objects. Would it suit your need ?

[michaelezra]

Hi Hombre!:) I think UI should have a single instance of the tool with sliders, etc. in the Toolbox. Multiple local edits should be possible, each editable when its anchor point is selected in Preview panel.

If I got point 1 correctly - this would lead to ALL (except not raw & transform) RT edits available for each local edit?! That would be exquisite! In such case, it may be better to Enable local edit mode first - this will make local edit anchors visible, editable and those double arrows on the sliders.

[Hombre57]

@michaelezra No, that would be the other way round : all slider could have unlimited amount of local edits, each different. E.g. in the Exposure tool, you could have a different set of local edits for the Contrast slider and for the Saturation slider.

Of course, local editing support for a slider would not be automatic, but the mechanism should be simple enough to manually implement it in most sliders.

[michaelezra]

When doing gradient and elliptical adjustments in ACR I frequently combine e.g. exposure, highlight adjustments and contrast, even clarity in a single anchor. If I understand correctly what you are proposing, next to every (applicable) slider there will be the local adjustment brush button. When it is clicked, all anchors for that specific slider would become visible and editable. The downside is that it is difficult to visualize all local adjustments made as they are tied to million RT sliders. I feel it would improve usability a lot if local adjustments are essentially overlayed RT profiles (modulated by mask, gradient, etc) with full arsenal of RT sliders. This relates to point 2 in your list above. This will greatly minimize number of anchors user has to place and work with, as each anchor represents a group of adjustments - a pp3. Do you think this is doable? Of course having xmp / structured format would make it much easier to structure such pp5? files:)

[Hombre57]

This relates to point 2 in your list above.

The effect created by having layer or by having heterogeneous (i.e. variable) parameters would not be the same, so I wouldn't mix point 1 and 2. What you want is adjusting several sliders for each modifier. I have to think about it, to be able to have both solutions available (yours and mine). But we might expect a lot of Sliders to be locally editable, so you'd have to Add new slider on the modifier object instead of seeing ALL of them already displayed. Is that ok ?

A single button would let you see all modifiers so you can attach a new slider to them or only display the one where the current slider is attached to.

Of course having xmp / structured format would make it much easier to structure such pp5? files:)

Probably, but who will do that ? :wink:

[Beep6581]

but then the pp3 will have to handle binary objects

Compressed and base-64 encoded text blob? Maybe one of our sibling projects has a solution for this, IIRC darktable. If editing a 10 megapixel image, would that mean RT would have to store a 10 megapixel mask?

My dream for RT would be for it to allow me to increase or decrease the exposure of some parts (basically dodge and burn) and to warm up some parts (painting a warmer white balance, color toning or whatever else which will warm up the colors).

[Hombre57]

If editing a 10 megapixel image, would that mean RT would have to store a 10 megapixel mask?

With the help of some optimization, not necessarily. Of course, one of the question will be the choice of the alpha channel bit depth. 8 bits int ? 16 bits int ? 16 bits Half-Float ? 32 bits float ? Since the whole engine use float numbers, I'd suggest using a 32 bits float channel.

We're still hijacking Jacques' issue btw.

[Hombre57]

My dream for RT would be for it to allow me to increase or decrease the exposure of some parts (basically dodge and burn)

Dodge and burn is only available when merging 2 layers, so point 2 in my comment above.

and to warm up some parts (painting a warmer white balance, color toning or whatever else which will warm up the colors).

This will be possible with point 1.

[Desmis]

I just made another change of shape detection algorithm for lightness and chrominance. Now I also take into account "hue". The answer of the algorithm is now very close to that of "Nik Software" (which can be coupled free with Photoshop.) The algorithm is activated as soon as scope is less than 20. Attention algorithm does not take into account the chromatic noise, which of course will disrupt the system. It is therefore recommended to treat the noise before.

With this algorithm, I think it is not, in most cases, need to finely delineate the area to be treated

:)

[Desmis]

I just fixed 2 bugs : 1) when mouse sensor was out of preview, in some cases leads to crash 2) best algorithm for lightness I hope this 2 fixes solved these bugs :)

[heckflosse]

@Desmis

Hello Jacques,

I hope you don't mind that I currently have not much time to look at your changes. When RT5 is releases, I'll take a look again!!!

Ingo

[Desmis]

@heckflosse

Ingo No problem. Thank you again :)

The actual "algoritm code" contains many arbitrary values (thresholds filters, ...). Of course I can make parameterizable via "options", but before I want users assess the new algo. :)

[Desmis]

There is still a bug, I think due to hueref, chromaref, lumaref between improccoordinator.cc and dcrop.cc

[Desmis]

I just push a change "merge with master"

1) I am working to another improvment for shape algorithm detection 2) actually we can only work with 5 spots, but there is no problem to have 10 or more. I'll just configured code to allow more spot. 3) I think I found the bug, it is "only" a pipeline problem, with refreshmap, and instructions "todo & someting". But I do not know exactly how to do :)

[Desmis]

1) above, is done. I hope no error :)

[Desmis]

I just posted a change with: 1) improving the shape detection algorithm for part Retinex. I used the same principle as in lightness/contrast/chroma

2) clean up the code

3) allow the user to choose the number of spots. I am not come to establish a dynamic system, not the dynamic mode is impossible, but the monitoring / selection that I have set up (slider) is very difficult to operate. You can choose the number of spots in "options" by changing the variable Nspot. By defaut the value is 3, you can select between 1 (stable) and ??. Warning! This will bring probable incompatibilities with previous installations including the *.mip

4) I'm still not arrived to fix the bug that causes crashes. I think it is a pipeline problem, with a LUT "gamma2curve" with a passage of wrong settings (nan). Knowing that this LUT is never used by locallab. I left the "mutex" in place, even if used incorrectly (I do not know how!), as a precaution, but if we can solve the crash, may be able to do we remove. :)

[Beep6581]

Compiles and runs fine here. I tried to reproduce the crash someone in the forum experienced when using 5 spots, but it didn't crash.

I did find that each spot has an affect on the other spots even when they don't touch. PP3 file with 5 spots https://filebin.net/0ie7hi3np0vbf7bl - remember to set the options file to 5 spots too. Scroll through all 5 spots, and you will see how all of them change when you don't change anything other than selecting a different spot.

I would find it useful if I could locally adjust exposure using a slider or preferably a curve, and be able to locally change white balance. Also the ability to control the edge falloff would make the locally edited areas blend in more nicely - currently edges are quite sharp.