perspective transform not giving same result

[Msabih]

Hi,

I checked perspective transform using same source and destination points and the perspective transform should be giving the same array back but for some reason, it does not. I wanted to calculate a metric using perspective transform and the metric gave different results when compared to opencv's perspective transform.

Here is my code to reproduce. ` import torch import torchgeometry as tgm import numpy as np import cv2

read the image with OpenCV

image = cv2.imread('./data/bruce.png')[..., (2,1,0)] print(image.shape) image = np.array(image, dtype=np.uint8) img = torch.from_numpy(image.transpose(2, 0, 1)) img = torch.unsqueeze(img.float(), dim=0) # BxCxHxW print('img size ', img.size())

the source points are the region to crop corners

points_src = torch.FloatTensor([[ [125, 150], [562, 40], [562, 282], [54, 328], ]])

the destination points are the image vertexes

h, w = 372, 600 # destination size points_dst = torch.FloatTensor([[ [125, 150], [562, 40], [562, 282], [54, 328], ]])

compute perspective transform

M = tgm.get_perspective_transform(points_src, points_dst)

warp the original image by the found transform

img_warp = tgm.warp_perspective(img, M, dsize=(h, w)) ##################THIS SHOULD GIVE EQUAL AS TRUE ############# print('images are equal or not ', torch.equal(img, img_warp))

convert back to numpy

image_warp = img_warp.byte().cpu().numpy()[0].transpose(1, 2, 0) print('image warp size ', image_warp.shape)

draw points into original image

for i in range(4): center = tuple(points_src[0, i].long().numpy()) image = cv2.circle(image.copy(), center, 5, (0, 255, 0), -1) `

[edgarriba]

@Msabih if you follow the example, your destination points should something like:

points_dst = torch.FloatTensor([[
    [0, 0], [w - 1, 0], [w - 1, h - 1], [0, h - 1],
]])

where w, h are the destination height and width.

Besides, I have recently introduced crop_and_resize which already implements this same functionality. Please, try both approaches and let us know. I will update the tutorials very soon.

[Msabih]

@Msabih if you follow the example, your destination points should something like:

points_dst = torch.FloatTensor([[
    [0, 0], [w - 1, 0], [w - 1, h - 1], [0, h - 1],
]])

where w, h are the destination height and width.

Besides, I have recently introduced crop_and_resize which already implements this same functionality. Please, try both approaches and let us know. I will update the tutorials very soon.

Does it matter ? From my understanding, if the source points and destination points are identical then perspective transform should output the same image. Thats what I was trying to test here i.e giving the same source and destination points and checking torch.equal(). In opencv, this test works, if I give same source and destination points and use np.array_equal(), the two arrays are equal.

[Msabih]

@edgarriba

Here is an example to demonstrate what I am pointing towards. Could you please run the code and check why the difference of warp image and image with same source and destination points for tgm gives me two lines whereas in opencv, it gives zero image ?

import numpy as np
import maplotlib.pyplot as plt
import cv2
import torch
import torchgeometry as tgm

## initialize the mask to be used in tgm ###
npimg1 = np.zeros((1, 1, 64, 64), dtype=np.uint8)
npimg1[:, :, 29:31, :] =1
image = torch.from_numpy(npimg1).float()

points_src = torch.FloatTensor([[
    [63, 0], [0, 63], [63, 63], [0, 0],
]])

points_dst = torch.FloatTensor([[
    [63, 0], [0, 63], [63, 63], [0, 0],
]])

M = tgm.get_perspective_transform(points_src, points_dst)
img_warp = tgm.warp_perspective(image, M, dsize=(64, 64))

np_image = image.cpu().numpy()[0][0]
np_wrp_image = img_warp.cpu().numpy()[0][0]

diff_img = np_image - np_wrp_image
plt.imshow(diff_img)
plt.show()

### Do the same in opencv ###

npsrc_pts = np.array([[63, 0], [0, 63], [63, 63], [0, 0]], dtype=np.float32)
npdst_pts = np.array([[63, 0], [0, 63], [63, 63], [0, 0]], dtype=np.float32)

M = cv2.getPerspectiveTransform(npsrc_pts, npdst_pts)
warp_image = cv2.warpPerspective(npimg1[0][0], M, dsize = (64, 64))

diffimg = warp_image - npimg1[0][0]

plt.imshow(diffimg)
plt.show()

EDIT : The results are okay if I do torch.round the output warped image...........

[edgarriba]

ok, sorry I misunderstood what you were trying to do int this test. To compare tensors I suggest to use torch.testing.assert_allclose.

[edgarriba]

@Msabih should we close this in favor of https://github.com/arraiyopensource/kornia/issues/133 ?

[Msabih]

@Msabih should we close this in favor of #133 ?

Yeah sure, this should be closed because this was'nt really a wrong result. It was just off by floating precision. #133 however looks like a genuine case as there the transform is blank.

[edgarriba]

@Msabih thanks. Yes, would appreciate the effort for solving this. Maybe by designing a toy unit test will help.