Understanding `kornia.geometry.homography_warp` coordinate system

senyai commented 1 month ago

Describe the bug

Hello! This is my first time with kornia. I started with homography_warp function. Let's say I have an image of size 8x16 and I want to flip it with this function. Not because I want to flip an image, but because I want to understand the math behind it.

To flip an image along Y axis I used this matrix:

  (1.0, 0.0, 0.0),
  (0.0, -1.0, 8.0),
  (0.0, 0.0, 1.0),

Unfortunately the output image is not only flipped, but also shifted 1px:

Reproduction steps

from torch import zeros, tensor, float32
import kornia

image = zeros(1, 1, 8, 16)
image[0, 0, 2:6:2, 2:14] = 1.0
image[0, 0, :, 2] = 1.0
mats = [(
    (1.0, 0.0, 0.0),
    (0.0, -1.0, 8.0),  # why 7 works?
    (0.0, 0.0, 1.0),
)]

mat = tensor(mats, dtype=float32)
out = kornia.geometry.homography_warp(
    image,
    mat,
    (8, 16),
    normalized_coordinates=False,
    normalized_homography=False,
)

print("input:")
for line in image[0, 0].numpy().tolist():
    print([round(val, 4) for val in line])

print("output:")
for line in out[0, 0].numpy().tolist():
    print([round(val, 4) for val in line])

Expected behavior

And if I replace "8" with "7", I see 99.99% correct image. My assumption is that top left pixel's top left corner has coordinates (0,0) and bottom right corner of that pixel has coordinates (1,1). If that is correct, then my initial flip matrix is correct and there's a bug. If that assumption is incorrect, then coordinate system should be mentioned in the documentation.

Environment

Collecting environment information...
PyTorch version: 2.3.1+cpu
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Fedora Linux 39 (Thirty Nine) (x86_64)
GCC version: (GCC) 13.3.1 20240522 (Red Hat 13.3.1-1)
Clang version: 17.0.6 (Fedora 17.0.6-2.fc39)
CMake version: version 3.27.7
Libc version: glibc-2.38

Python version: 3.12.4 (main, Jun  7 2024, 00:00:00) [GCC 13.3.1 20240522 (Red Hat 13.3.1-1)] (64-bit runtime)
Python platform: Linux-6.9.7-100.fc39.x86_64-x86_64-with-glibc2.38
Is CUDA available: False
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: GPU 0: NVIDIA GeForce GTX 970
Nvidia driver version: 555.58.02
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Address sizes:                        46 bits physical, 48 bits virtual
Byte Order:                           Little Endian
CPU(s):                               12
On-line CPU(s) list:                  0-11
Vendor ID:                            GenuineIntel
Model name:                           Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz
CPU family:                           6
Model:                                63
Thread(s) per core:                   2
Core(s) per socket:                   6
Socket(s):                            1
Stepping:                             2
...
Versions of relevant libraries:
[pip3] mypy==1.9.0
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.24.4
[pip3] torch==2.3.1+cpu
[pip3] torchaudio==2.3.1+cpu
[pip3] torchvision==0.18.1+cpu
[conda] Could not collect

Additional context

No response

senyai commented 1 month ago

I have found one of the roots of this issue: https://github.com/kornia/kornia/blob/969169bc2aebc14f5c82abdefd3c85c3d31b62f0/kornia/geometry/conversions.py#L1037-L1038 if you subtract 1 from width you get incorrect transformation. After removing -1 my testcase showed correct output. Alas, when passing align_corners=False that transformation is still incorrect, but 0.5 pixels incorrect, and not 1.5 pixels incorrect 👍🏻

lappemic commented 1 month ago

Thank you @senyai for your input. We really appreciate your interest in using Kornia.

To ensure we can best support you and other contributors, we'd like to kindly suggest a few channels for future inquiries (also see here:

Our GitHub Discussions page is a great place to connect with the community and ask questions: Kornia GH Discussions
You can join our Slack workspace, where you can interact with our core contributors and fellow community members: Join Kornia Slack
The PyTorch Discuss forum is another excellent resource. Feel free to use the #kornia tag there: PyTorch Discuss

These channels help us maintain organized and efficient communication.

Looking forward to seeing you in one of these spaces or in other issues you encounter of course!

kornia / kornia