search

spillerrec / Overmix

Automatic anime screenshot stitching in high quality
http://spillerrec.dk/category/software/programs/overmix/
GNU General Public License v3.0
191 stars 14 forks source link

Implement rotation+scaling transform #153

Open spillerrec opened 4 years ago

ZenoArrows commented 1 year ago

I see you're considering motion vectors and log-polar transform, but have you considered SIFT or SURF? Those are scaling+rotation invariant feature detectors and are used to stitch together panoramas. Overmix could utilize OpenCV for an implementation of those algorithms.

spillerrec commented 1 year ago

I did look at the feature based methods available in OpenCV, but I didn't go with them for two reasons. First, I want to avoid big dependencies like OpenCV if possible and secondly I would like to learn how these methods actually work in depth instead of just using them. The log-polar transform seemed simpler, so that was the direction I took. But I realize I don't have the time/energy anymore to do that (also considering other stuff I want to mess with) so I might just try it out. If I didn't actually got the log-polar transform working, I don't quite remember but I did get pretty far.

ZenoArrows commented 1 year ago

I might be interested in trying my hand at an OpenCV-powered aligner as an optional dependency. The more options for alignment the better right?

spillerrec commented 1 year ago

Feel free, I will not promise that I will release a Windows build with it, but if it is an optional dependency that cmake will enable automatically if you already have the OpenCV the ones building it from source will benefit. (I know there are a couple.)

You need to make a class that implements the AComparator interface: src/comparators/AComparator.hpp findOffset is the main method you want to implement, with a1 and a2 being the alpha planes to img1 and img2 and hint being an optional estimate you can safely ignore. I believe findError returns the error for a specific offset using the distance metric which is used for findOffset, I don't remember exactly what it is for. You could add a rotation and scale option to ImageOffset if you want to experiment with that.

For adding it to the interface see how the others are defined in these two files: interface/gui/configs/ComparatorConfigsInternal.hpp https://github.com/spillerrec/Overmix/blob/master/interface/gui/configs/ComparatorConfigs.cpp In short, implement AAlignerConfig and add it as an option in ComparatorConfigChooser::p_initialize(). I don't like UI stuff so I tried to make it as simple as possible.

ZenoArrows commented 9 months ago

I've been experimenting a bit with the feature detector-based stitching algorithms in OpenCV before integrating anything into Overmix. I've found that it's pretty good at stitching together static images, but it struggles with stitching together animations as it can't differentiate between camera movement and character movement. Also since it's meant to stitch together images with a larger margin for distortion it often tries to warp the image or rotate it in odd directions.

So I think you were on the right path with the log-polar transform, since we're more interested in minor adjustments and have a much smaller error margin between each image. Did you happen to upload your work somewhere? I'd be interested in making an attempt to work on it.

spillerrec commented 9 months ago

I just pushed it to a branch which also contained a bunch of other random stuff. This looks to be the code I was experimenting with: https://github.com/spillerrec/Overmix/blob/vulkan-test/interface/main-align.cpp I don't think I made any code to actually rotate the images, I want to move to a GPU only workflow, but I haven't quite done the groundwork yet.