Make perspective correction not crop the image

[Beep6581]

Originally reported on Google Code with ID 207

What steps will reproduce the problem?
1. Take image which needs perspective correction
2. Correct it as needed
3. Part of the image gets out of the image boundary and cannot be recovered.

What is the expected output? What do you see instead?

I would like to have possibility to extend image boundary, so the cropped part of the
image is not lost or at least to have possibility to scale image down, that all image
data fit's the boundary.

What version of the product are you using? On what operating system?

RT3 alpha, all platform.

Please provide any additional information below.

Perspective correction is one of the most important developments for me as  I do photograph
a lot of architecture. Unfortunately this issue - which causes that the part of the
corrected image is croped beyond recovery, limits usefulness of the feature.

I attached example of uncorrected and corrected image to illustrate how the part of
the image is cropped by correction. 

Reported by michal.thoma on 2010-09-03 16:45:59

[Beep6581]

I should note that this is enhancement request, not defect though I couldn't find appropriate
select control when reporting.

Reported by michal.thoma on 2010-09-03 16:51:12

[Beep6581]

Reported by rinni@gmx.net on 2010-09-04 13:20:04

• Status changed: Accepted
• Labels added: Type-Enhancement
• Labels removed: Type-Defect

[Beep6581]

Hello, according to me I see only an uncorrected sample. That said, I don't know if
it is possible what you want. Perspective correction implies a correction of the image
- horizontally, vertically or both - This means there is always 'something going on
at the borders of the image', by definition as far as I can tell. 

Did you try to uncheck the Fill out option or whatever that is called in English? That
way the whole image is kept, but you'll see black borders of course. 

Reported by paul.matthijsse4 on 2010-09-04 20:47:00

[Beep6581]

Hello, sorry, corrected image is back, I deleted that accidentally. On this image you
can see how the top part of the image is cropped outside the boundary. It's no way
necessary. To illustrate this I include image from Photoshop's Lens correction tool
which is very similar to RT perspective correction. Though it has slider to scale image
down and thus recover even the part, which gets cropped correction. 

Reported by michal.thoma on 2010-09-04 22:57:41

[Beep6581]

I see what you mean now, PS seems to do a better job here.

Reported by paul.matthijsse4 on 2010-09-06 07:31:10

[Beep6581]

Maybe we could allow the crop tool to extend outside the image and fill in the parts
that are outside the picture with a custom color or 'transparancy'?

Reported by janrinze on 2010-09-06 10:42:42

[Beep6581]

As far as I'm concerned I don't need transparency. In most cases you want to crop the
black/transparent parts anyway, in rest cases you want to clone/paint something there
(sky or so) and in such case doesn't matter if it's black or transparent.

Anyway I think there are two options to address this-

1-possibility to extend crop outside image boundary
2-possibility to scale image down that it fully fits the boundary

Reported by michal.thoma on 2010-09-08 17:13:04

[Beep6581]

It is too mild to call this an "enhancement". Perspective control that crops a perfectly
useful part of the picture is a bug. And since perspective control is so often used
by architecture photographers I think this bug should be prioritized. My guess is that
it is not so hard to fix.

The typical use case when photographing architecture is that you stand on the ground
and need to point the camera upwards somewhat to get the whole building inside the
frame. To get the building stand straight up on the final picture you need to apply
perspective control, in this typical case tilt the "picture plane" such that the top
comes closer and bottom farther away. RawTherapee can do this - *but* - it crops away
the top part, meaning that you very often lose the top of the building.

Perspective control should not crop anything, just fill the "empty" space with black
or transparency, just like RT does for the bottom half of the picture in this case
(it is only above/below/left/right of the middle axis the picture gets cropped).

Reported by torger@ludd.ltu.se on 2010-12-04 22:20:38

[Beep6581]

Here's a patch that fixes the problem. However, it could help from some tuning, it seems
like the "autocrop" function does not work with this patch.

Reported by torger@ludd.ltu.se on 2010-12-05 00:21:48

[Beep6581]

Seems to me that a more thorough fix of all transform aspects (rotate has the same problem
etc) need to be made.

A proper fix must support auto-increasing the canvas size. My quick-fix does not increase
canvas size so the overall picture is shrunk (in this particular case you could argue
that it is the right thing to do -- to avoid upsizing pixels, but most people will
probably want average scale of 1.0 meaning that some pixels will be shrunk some upsized),
also happens for rotate (which still crops the picture).

Reported by torger@ludd.ltu.se on 2010-12-05 08:13:34

[Beep6581]

(deleting image attachments as we have run out of attachment space)

Reported by wyatt.olson on 2010-12-06 16:04:09

[Beep6581]

I would like to to try your patch, though can't import it using mercurial. On hg import,
mercurial aborts with this "abort: bad hunk #4 @@ -532,7 +532,7 @@". Even in the case
I update to the revision 95ffc835e5e1 against which revision was created...

This is probably the reason why nobody pushed this upward... Can you check and recreate
the patch?

Anyway I think resizing the corrected image down is perfectly enough. Even the Photoshop
Lens Correction Filter does like that. Decrease in the standard cases is from 10-20
% which is quite acceptable. Also the detail in stretched area of the image would get
even more blurry if enlarged more.

Reported by michal.thoma on 2011-01-10 16:01:43

[Beep6581]

Please also take a look at issue 455 (http://code.google.com/p/rawtherapee/issues/detail?id=465)

Reported by michaelezra000 on 2011-01-10 16:09:35

[Beep6581]

I think that the user shouldn't have to worry about the canvas size, but only select
the behaviour of the function: there should be a slider with a 0(only compress pixels)
to 1(only stretch pixels) range, plus an "Autofill" option (like the one of the rotate
tool). If you set it off, you'll be able to use the entire result of the autoresized
canvas.

Reported by natureh.510 on 2011-01-13 02:11:17

• Labels added: Milestone-3.1-b2

[Beep6581]

Reported by entertheyoni on 2012-01-02 19:40:56

• Labels added: Priority-Low
• Labels removed: Priority-Medium, Milestone-3.1-b2

[Beep6581]

Issue 455 has been merged into this issue.

Reported by entertheyoni on 2012-01-02 19:43:05

[Beep6581]

Then disable 'Auto-fill'

Reported by heckflosse@i-weyrich.de on 2015-06-01 20:57:08

[Beep6581]

I have deleted my comments and merged them.

I expirienced this. I wanted to use perspective correction to fix imbalanced composition,
instead perspective correction wasted half of the space which I gained.

I do not like it.

A simple (?) workaround will be to introduce the "scale" tool - a slider for magnification
of the image.

For some unknown reason users of ACR6.7 (did not check recent versions) were fine with
decrease of resolution - the perspective correction looks exactly like in RT, it crops
the image and the user may compress it back with "Scale" slider. Then user crops the
meaningful part of the image - which has smaller resolution than one of RAW file.

>Then disable 'Auto-fill'

It does not work that way. The original issue says same thing. Also: if there is Autofill
we can program it to have user provided magnification.

Reported by pinhuer on 2015-06-01 20:59:01

[Beep6581]

#20 was for #17 ++ which are deleted meanwhile

Reported by heckflosse@i-weyrich.de on 2015-06-01 20:59:40

[Beep6581]

I don't understand why disable 'Auto-fill' doesn't work for your case. Please explain

Reported by heckflosse@i-weyrich.de on 2015-06-01 21:02:49

[Beep6581]

Even with auto fill disabled some parts of the image are cropped as is apparent if you
ever tried that. It's easy as that.

Reported by michal.thoma on 2015-06-01 21:05:36

[Beep6581]

Sorry, my fault. I read too fast and read 'Distortion' instead of 'Perspective' :(

So we would need to be able to define the axis out of centre.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-06-01 21:11:50

[Beep6581]

>So we would need to be able to define the axis out of centre.

That does not exactly help. The perspective correction inevitably extracts the things
which User might need. The simpliest solution is still "Scale", even more so when it
is already implemented - we just need to make it manual.

Reported by pinhuer on 2015-06-01 21:15:37

[Beep6581]

Pinhuer, perhaps I miss something, but assume you move the perspective Horizontal slider
to -100 and the axis would be at left border, wouldn't that solve the Issue?

Reported by heckflosse@i-weyrich.de on 2015-06-01 21:22:03

[Beep6581]

michal.thoma: Would #27 solve the Issue in your opinion (of course it would be same
for perspective vertical slider and not only for a value of -100 ;-) ?

Ingo

Reported by heckflosse@i-weyrich.de on 2015-06-01 22:10:22

[Beep6581]

No. Please check comment no. 8 which perfectly explains what is needed.

Reported by michal.thoma on 2015-06-01 22:10:28

[Beep6581]

In short - we either need ability to scale the image down to bounding box or to be able
to extend the bounding box to fit all evene marginal parts of the image. And then again
crop what we really need.

Reported by michal.thoma on 2015-06-01 22:14:13

[Beep6581]

Hmm, but with my suggestion from #27, nothing would be cropped and user could choose
crop and scale. Do I still miss something?

Ingo

Reported by heckflosse@i-weyrich.de on 2015-06-01 22:54:19

[Beep6581]

Though, #21 is not imprtant. I'll not implement this feature because I've enough in
my todo list. I'll let this one open for pinhuer ;-)

Ingo

Reported by heckflosse@i-weyrich.de on 2015-06-01 23:08:20

[Beep6581]

Well maybe I do not understand the merit of supposed solution in #27 but I think just
shifting axis will be partial help only. Anyway the patch linked #9 did work for me,although
it was not incorporated and now it can't be. Anyway still hope for this can be done,
as for the architecture photo this one is very helpful.

Reported by michal.thoma on 2015-06-01 23:14:57

[Beep6581]

michal.thoma, I'll try patch from #9 tomorrow and post a new patch which applies to
tip if it still works.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-06-01 23:18:25

[Beep6581]

Good luck and thank you for focusing on this! Michal

Reported by michal.thoma on 2015-06-01 23:22:38

[Beep6581]

Hah, I reworked the patzch from #9 to apply to tip. And it already works as I suggested
in #27 (moving axis). I'll clean and post it tomorrow :-)

Ingo

Reported by heckflosse@i-weyrich.de on 2015-06-01 23:35:47

• Status changed: PatchSubmitted

[Beep6581]

Wow, great! Seems like this issue was waiting for you. Looking forward to check it when
it's applied!

Reported by michal.thoma on 2015-06-01 23:58:42

[Beep6581]

Here's the patch from #9 reworked for tip. At extreme values of perspective correction
it still crops the image but I think these values are not of practical use. There are
also some issues when perspective correction is combined with other corrections (e.g.
distortion correction). It also breaks compatibility.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-06-02 11:37:15

---

• _Attachment: issue207_01.patch_

[Beep6581]

Just tried new patch and in sense of cropping it's definitely a great improvement, especially
for typical usage scenario! 

Still there is some cropping happening, especially when use with combination of rotation
and both vertical and horizontal perspective. See two linked images of corrected and
uncorrected as below:

https://docs.google.com/file/d/0ByGTquk6g5BYX0JOYXZKTXk5QTg/edit?usp=drivesdk
https://docs.google.com/file/d/0ByGTquk6g5BYSDRkLVVtX1BFa0k/edit?usp=drivesdk

In this particular case the lost part of the image on the top is actually for advantage,
but in quite typical case scenario (like shiny tip of temple tower or similar) this
can actually hurt the composition badly. So yes, the crop is marginal but in this case,
every pixel matters.

So still I see space for some more comprehensive solution (like scale tool discussed
above) which would help rotate and distortion tools as well.

Anyway I'm sure that this would work for most of usage cases, so thank you so much
for the effort and great improvement! :)

Michal

Reported by michal.thoma on 2015-06-02 18:09:27

[Beep6581]

Michal, kudos for Anders who made the original patch from #9, not for me. I only reworked
the old patch to apply to tip ;-)

Tomorrow I'll take a look whether I can solve some of the remaining issues.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-06-02 22:22:05

[Beep6581]

Michal, after looking at the code, it seems that an additional scale slider in 'Lens
/ Geometry' tool could be the easiest solution. Possibly enabled only when 'Auto fill'
is off.

Ingo

Reported by heckflosse@i-weyrich.de on 2015-06-02 23:19:13

[Beep6581]

Patch: https://gist.github.com/ff0776d28ceb89b83ef6

[romandocent]

When this function will be added into "release"?

[scaldov]

Run into the same trouble. The top of the house is cutted after perspective correction. There has 6 years passed by! What I should do?

[TechXavAL]

@scaldov

Could you try the new «camera-based» way of correcting perspective (Perspective > Method: camera-based)? It's in the dev branch.

Try applying a positive «Vertical» perspective correction together with a negative «Vertical shift». «Auto-fill» must be unchecked (turned off).

Does this solve the problem? If so, maybe this issue can be (finally) closed while properly documenting it in Rawpedia.

[heckflosse]

Hmm, I don't understand the issue

You can disable auto-fill

and enable Auto-Crop

or crop manually

[TechXavAL]

As I understand it, the issue is with vertical perspective.

Let's say you have a building like this one, that you wish to correct its vertical lines:

So you play with the «Vertical» slider until you get what you want (in this example, it's overdone to better see the problem):

As you can see, the lower half the corrected perspective shows black borders, but that's not the case in the upper half, where the building gets cropped no matter what you do (by default).

My point was that playing with the «Vertical shift» slider, you can almost get the cropped upper part of the image back. There's still a decision to make: if you want the whole upper part of the image, you will usually loose some of the lower part. So you have to choose what to loose from the image, but now at least you have the opportunity to save the building:

[Lawrence37]

Horizontal and vertical shift should be sufficient in many cases, but in extreme cases, it would be nice to enlarge the canvas. By the way, when shifting, use the adjusters in the post-correction adjustments section. The ones in the correction section are used when the image is not aligned with the lens's center (tilt-shift lens for example).

[th555]

Why not follow KISS principles, and simply always enlarge the canvas to the required size so that data never gets lost? One can worry about cropping separately, using the crop tab. A rotation slider in the perspective tab seems superfluous as well, since the existing Rotate option can be used together with perspective correction just fine.

[Lawrence37]

@th555 It's a good feature to have and simple in principle, but it's a challenge to calculate how much expansion is required, especially when distortion correction is involved. The rotations are applied at different stages in the transformation and will have different results in some situations.

[pappasadrian]

Hello,

I've ran into the same issue, where due to (arguably somewhat extreme) vertical and horizontal perspective changes, a lot of my theme is out of frame. This happens despite auto-fill being turned off. By adjusting vertical shift, I can get the bottom or the top cropped parts back in frame, but not everything all at once. A silly workaround that I am applying, is to export the image with two different vertical shift settings, and then joining the two halves in gimp - really not ideal. An option to resize the canvas, or enlarge the canvas to ensure that every pixel is inside (even if this leaves a lot of empty space) would be highly appreciated.

Thanks!

[th555]

Another simple-to-implement (I'm guessing without having seen the code, but it's in similar vein to the shift and rotate options) solution would be to include a scale slider in the perspective tab, so the transformed image can be shrunk as needed.