Enhance perspective correction

[RawConvert]

Following my post here - https://discuss.pixls.us/t/feature-suggestion-converging-verticals/5670 and more recent discussion here - https://discuss.pixls.us/t/perspective-conversion/7206/9 this issue puts the suggestion in the correct place for possible consideration.

In summary, the current RT correction process is rather slow, involving iterations of horiziontal or vertical correction together with use of the straighten tool. What would be good is a tool that allowed correction of converging verticals (or horizointal correction) in one action, for example by the user making two lines which would then be processed to parallel.

[Beep6581]

The way perspective correction works in Autopano Pro/Giga is amazingly easy to use and efficient. Basically it's as @RawConvert described it in the last line - you draw two (or more, but two can be enough) lines, and these are made parallel. They can be horizontal, vertical, or both (4 lines). https://youtu.be/voVBTiRxcLk?t=39

[aferrero2707]

I implemented something similar in PhotoFlow, but the code is actually derived from Darktable: https://www.youtube.com/watch?v=RCbpjyrtXLc

Here is where the perspective correction is computed: https://github.com/aferrero2707/PhotoFlow/blob/stable/src/vips/perspective.cc#L226

I can give you some hints if you are interested.

[chaevvi]

It is not always convenient to carry out two lines. I like the visual way of transforming perspective, as in the GIMP editor - pulling the image with the mouse. At the same time, you focus on a rectangular grid on the image. I propose to implement this method in RawTherapee. It is also possible to implement other convenient ways - at the choice of the user. The current method of correcting the perspective requires a large number of actions, which is inconvenient.

[Bezierr]

I support chaevvi in this. Perspective correction for anything more complex than a slight tilt of the camera is something that I, too, do in Gimp. I like the way it is done there, with one exception: when choosing "forward" correction, the grid should not move with the image but remain stable (i.e. rectangular), so the image can be adjusted to the grid. So, it would be great if this could be done directly in RT.

[chaevvi]

Regarding Bezierr comment: when adjusting the perspective in GIMP, you need to enable and select the grid size on the image. Then the undistorted rectangular grid and the mesh deformed during perspective correction will be displayed at the same time.

[Bezierr]

Ah - didn't know that (I don't use Gimp all that often). Still, I think the following logic would be, ehm, more logical:

• Forward correction: Grid stands, image moves.
• Backward correction: Image stands, grid moves.

[Beep6581]

1. Have you tried the 2-lines method, and if so, in which program?
2. Could you show a video (screen recording) where using the grid is easier than drawing two lines? Thanks.

[RawConvert]

In my experience, it's usually pretty obvious where the key lines are for correction, and drawing them in is therefore quick and easy, and the rest is automatic and mechanical, the computer does it. In the rare case where you have a strange image which still doesn't look quite right even though its lines are geometrically correct (or at least as correct as your lens aberrations allow), I imagine the devs will not remove the current tools, so one could still adjust it to taste.

Having adjustable grids in forward and backwards modes sounds complicated to use and harder to build.

Though the current tools are slow to use, I guess you've realised @chaevvi & @Bezierr that you can slide in a crop edge in order to check accuracy.

[chaevvi]

1. Unfortunately I don't know how to make video from screen. 2. In GIMP, I just move the picture with the mouse to get the desired effect, directing the initial stationary grid. 3. Method two lines I have not tried it and not even know about it. Probably there are many more convenient methods of perspective correction. My suggestions are based on what I've already tried. I will only be happy if there are other correction mechanisms in addition. However, the dynamic perturbation of the picture the mouse I think it is easier, more accurate and more flexible than drawing lines and then, if necessary, to redraw them again. To draw lines you need to aim correctly.

[Beep6581]

So you have not tried it, you don't know how it works, but you are of the opinion that it's no good and that GIMP's antiquated method is better, did I get that right?

[Bezierr]

I haven't tried the "two lines" method, either, since none of the software I use offers it.

The big drawbacks of the "conventional" perspective correction method, as implemented in RT, but also in LR and others, are that a) the image is severely cropped (and I have no control over where this cropping should occur), and b) I don't have any control over overall compression or stretching, frequently resulting in shapes that are taller or wider than in reality. So in my opinion this simple tool is only useful for slight everyday perspective corrections when I didn't hold the camera quite straight; for everything else, the Gimp method, antiquated or not, works very well for me.

Regarding "two lines": Do I have this control? Will it still work when horizontal and vertical distortion and rotation need to be corrected in the same image?

[chaevvi]

Two-line perspective correction can be very effective for some shooting scenarios. But not everyone. For example, for landscapes and other artistic compositions the choice of such lines is not always obvious. Moreover, it is not always necessary to do exactly parallel lines of geometric perspective. Sometimes this greatly worsens the frame, for example, the volume effect. Therefore, a more universal method is needed as the main one, and additional methods are needed for more specific applications. As for my lack of experience in applying the method of two lines, the first photo in the discussion is enough for me https://discuss.pixls.us/t/feature-suggestion-converging-verticals/5670 and a good spatial imagination. In addition, with the correction of prospects, I worked for a while and have some practical experience.

[Beep6581]

Example 1: https://filebin.net/383qwo5xw7tyfjzh/vid-2018-04-22_19.00.29.mp4 Example 2: https://filebin.net/383qwo5xw7tyfjzh/vid2-2018-04-22_19.04.00.mp4

[chaevvi]

I agree - perpendicular lines is a good mechanism. As I already wrote, above-I highly appreciated it. I am in favour of implementing this wonderful mechanism. However, I like to watch and evaluate how the picture changes in dynamics when I move it with the mouse. In addition, there are pictures which when aligned in straight lines on them (the corners of buildings, boundaries, paths...) become unnaturally straight. Especially when shooting with a short-focus lens.

[Bezierr]

@Beep6581 , thanks for the examples. It's what I expected from the name, and clearly this is much better (easier to use, better results) than the current solution. I guess I could use it for 95% of my perspective corrections, and for the remaining 5% -- well, there is still Gimp.

[Hombre57]

In a GUI point of view, there is 2 ways to implement what you're talking about.

1. Freeform deformation : with this method, you could place some helper line in the image, showing actual angle, then pick one of the for corner of the image to deform it at will, with a visual feedback of the result (and updated angle for the helper lines), and possibly not totally suppressing the perspective. You would also control which part of the image would be stretched (becoming softer) and which would be shrank.
2. Automatic deformation : here you enter a Perspective editing mode to show the original image, place the 4 corners of a box (as shown by @aferrero2707) or the helper lines, and return to the Normal mode where RT would compute the deformation automatically and show the deformed image. Eventually, we could add an Amount slider to linearly interpolate between no correction to full correction, if one want to keep some perspective.

As I understand it, @aferrero2707 has a working algorithm with a 4 corners mechanism ready to be copied. Do you have some code to use for the 2 lines methods ?

[Beep6581]

Do you have some code to use for the 2 lines methods ?

No, the example is from Autopano Giga which is proprietary. I suspect Hugin could have code which does just that (the UI in Hugin works differently, placing two lines is not so simple, but aside from the UI the engine does the same job), but currently I cannot point you to it.

[Hombre57]

@Beep6581 At least I think I can find it myself in Hugin now, thanks.

Should the methods be cumulative, i.e. do you want a Method combo to still have access to the old method + new one(s) but only one at a time, or do you want to be able to apply all of them in a single shot ? I'd prefer enabling only one.

[Beep6581]

@Hombre57 great that you found the relevant section. All things considered, I vote for not-cumulative.

[RawConvert]

I'm not clear what it will look like following the two posts above, for example how many tools would there be once implemented. If you correct using two drawn lines, will this populate the existing horizontal, vertical and rotation tools? That is, if you wanted to tweak the rotation slightly, could you just go to that tool and alter the value a little? @Beep6581 @Hombre57

[RawConvert]

Some more on this - just my 2 cents - I know it's not me building it!... Answering my own question, I don't think there is a new tool or tools! I see it as analogous to the current straight line "icon". You draw the line, but how does RT see it? - in the history, it's a Rotation, presumably likewise in the PP3. The "fundamental" "tools" at play here are Rotation, Horizontal and Vertical. You can call the straight line icon a Tool, but fundamentally it's just a short-lived aid making it easy to set the Rotation tool. Similarly drawing two lines for perspective correction is a transient help function for setting up the three fundamental tools. Presumably the history and PP3 would show the changes to these three. Thereafter any further manual changes to horizontal, vertical or rotation are just adjustments that get rolled up when the pipeline is processed. And no changes to the PP3 structure. Hope this helps.

[Hombre57]

@RawConvert Yes, that helps, thought I don't know if using this feature with successive occurrence will lead to a predictable result. But we should go the way you just explained IMHO.

[chaevvi]

I agree that the implementation of perspective correction with the help of lines should be put as a priority. Will receive great benefits.

In this case, the lines can be conducted and not necessarily parallel.

This is a relatively easy job compared to the implementation in GIMP. And it will bring the greatest initial benefit to RawTherapee, given the small Speed of rawtherapee image transformations compared to GIMP.

I will look forward to the specified function! After its implementation and testing in practical examples it will be possible to discuss what else is needed for the effective correction of prospects.

I would like to have a similar tool for correcting distortion. For example, by specifying a rhombus shape from lines with diagonals approximately vertically and horizontally .

Taking into account the existing mechanism of rotation along the line all together will give a powerful tool to adjust the image.

It is important that the specified method (s) become a target and there are no distractions to other parties until it is.

[heckflosse]

@chaevvi

It is important that the specified method (s) become a target and there are no distractions to other parties until it is.

really?

[Beep6581]

It is important that the specified method (s) become a target and there are no distractions to other parties until it is.

https://www.paypal.me/rawtherapee