MelodYanglc
commented
2 weeks ago Hello, thank you very much for your interest in our work. Regarding your question, the explanation that needs to be made is that the algorithm still shows a little bit of boundary irregularity in its rectangularized image when dealing with stitched images with severely irregular boundaries. This is a problem that we have not solved. Meanwhile, even in the original DeepRectangling [https://github.com/nie-lang/DeepRectangling] article there is the same problem to be solved. However, here I can come up with several solutions as follows:
- Similar to DeepRectangling, the introduction of the mask loss constraint term will improve the boundary regularity of its rectangularized image. However, in our experiments we have eliminated the constraint because we are trying to explore a wider range of rectangular deformation processes. At the same time, strengthening the mesh constraint term will also improve the problem moderately. (e.g., coefficients in rows 15, 16 in the loss function.)
- For a more efficient solution, you might consider introducing optical flow estimation for further excess of content deformation. (e.g., CoupledTPS [https://github.com/nie-lang/CoupledTPS]), but this will also greatly limit the upper performance limit. In addition, diffusion model-based solutions perform more promisingly, but resource consumption is also a huge problem. (e.g., RecDiffusion [https://github.com/lhaippp/RecDiffusion]).
Our experiment on our own dataset found that when the vacant region of irregularly spliced images is too large, it leads to rectangularization results that still have a large defect out.