Feature request: Add tool for de-fishing images taken with fisheye lenses

[abrock]

I suggest adding de-fishing capabilities to RT. There already is a feature "Distortion correction" but that seems to be limited to the distortion exhibited by common rectilinear lenses.

This was taken with a 6.5mm f/2 lens:

And I want to have the option to convert it to this:

I implemented a demo how to defish an image using OpenCV to create the above example: https://github.com/abrock/defishdemo

I could give it a shot to implement it in RT, but I'd like to get a bit of feedback first if this feature would be accepted.

When undistorting a fisheye image there is one problem: The source pixels will be stretched more the further away from the center they are. This means that if I want to preserve the resolution in the center and (most of) the field of view I have to increase the size of the destination image. In the example above I scaled width and height by a factor of 3.45. Is this possible in RT? I think there also would need to be controls for that so the user can choose between increasing image size and reducing the resolution in the center.

[Lawrence37]

There's this closed feature request: #1843. In my opinion, a tool to convert from fisheye to rectilinear is reasonable.

I have wondered about changing image sizes. It would be useful for perspective correction for example. There are tools that affect the resolution - the resize and demosaic border - but they work very late or early in the pipeline.

[abrock]

I added a hard-coded fisheye un-distortion for testing / discussion: https://github.com/abrock/RawTherapee/tree/defish

Here's an image you can play around with: https://drive.google.com/drive/folders/1_DOu7Zm-_YjaRgid8okSnyvQ6rP6Gtm8?usp=share_link

If you want to use it on different images: De-fishing is only active if another geometric operation is active (e.g. perspective, CA etc.) so I set the distortion correction to 0.001 in the profile. The result is almost pixel-perfect identical with the result I get with my defishing-demo if I switch interpolation to lanzcos.

One problem occurs when zooming into areas not close to the center, then parts of the preview are black:

I think the reason is somehow the extreme distance between source and destination pixels. If I put the image into the queue and activate the queue I don't have that problem.

Questions:

• Should this be integrated into RT?
• What should the GUI look like? I think the user should select at least focal length and downscale factor, image size increase would also be nice if possible.

[Lawrence37]

There's another function that performs geometric transforms which (I think) is used for auto-crop and determining the size and location of the input region of interest. Adding the de-fish code there is required and may solve the black preview problem.

For the GUI, scale and size should be for the whole Lens/Geometry tool so that it plays nicely with Auto-fill and because it is logically useful for the other sub-tools.

[abrock]

I added a "Scale" slider to the perspective tool and managed to get it to work as intended, its setting is saved and loaded.

I also added a "De-fish" checkbox, but it's nonfunctional and I don't understand what I have to do to get it to work.

I also checked if the combination of fisheye undistortion and perspective correction works simultaneously by rendering a checkerboard parallel to the camera with blender. When I un-distort it in RT and change the perspective at the same time it still looks like a checkerboard captured with a rectilinear lens. See test-render.tif:

https://drive.google.com/drive/folders/1_DOu7Zm-_YjaRgid8okSnyvQ6rP6Gtm8?usp=share_link

Can you point me to the second geometric transforms code? Is there some kind of development mailing list / is this the main place for development discussion?

[Lawrence37]

I also added a "De-fish" checkbox, but it's nonfunctional

Non functional as in it doesn't toggle, notify the signal listener, or update the preview?

This is the other place: https://github.com/Beep6581/RawTherapee/blob/8047aded3ba4756233be0d4e833e535bdd6293f0/rtengine/iptransform.cc#L420

Is there some kind of development mailing list

I only use GitHub (and some pixls.us), but I can't speak for others.

[abrock]

It toggles and triggers a re-compute of the image, but it doesn't change the value of bool PerspectiveParams::camera_defish since I don't know how to connect the two.

I'll look into the ImProcFunctions::transCoord function, thank you

[abrock]

I put the de-fishing code into ImProcFunctions::transCoord and now the preview works as expected.

[Lawrence37]

You need to set/get the value in PerspCorrection::read and PerspCorrection::write

[abrock]

Nice, now the behavior toggles when I click the checkbox in the GUI. The value is also stored in the sidecar file as "CameraDefish=true", but when I close and re-open RT it doesn't load it correctly.

[abrock]

I managed to fix the issue with "De-fish" not being loaded correctly. I tried to add translation strings for English UK/US for "De-fish" and "Scale" but failed, can someone help me?

I'm very satisfied with the behavior now, it stores and correctly stores and loads the Defish and Scale properties I introduced, and it works nicely in combination with the other perspective adjustments. What do you think?

[Lawrence37]

Add new language keys and strings to default. It's the default language file written in American English that is then translated to other languages. I don't know why there is an English (US). English (UK) is generated from default using a script.

[abrock]

I did a couple more commits and got the strings working as far as I can see. Should I make a merge request? Should I squash the commits into one single commit in a new branch? Should I run tests to make sure I didn't break anything?

Thank you very much for helping me.

[Lawrence37]

Yes, please open a pull request. That will make testing and feedback easier. GitHub has an option to squash and merge, so squashing is not necessary. Just make sure the code is tidy and that it works as expected and handles any corner cases you can think of.