xlwangDev / HC-Net

[NeurIPS 2023] Fine-Grained Cross-View Geo-Localization Using a Correlation-Aware Homography Estimator
50 stars 5 forks source link

Transform for CVUSA. #1

Closed liguopeng0923 closed 1 year ago

liguopeng0923 commented 1 year ago

Hi Xiaolong,

I have reproduced the Spherical Transform in VIGOR. However, it isn't easy to get a normal result in CVUSA. Whether there are additional operations in CVUSA?

Best, Guopeng.

guanfang12 commented 1 year ago

Because standard panoramic images should have a 2:1 aspect ratio, and the images in CVUSA are cropped with a non-standard aspect ratio, as mentioned in Section 4.4 of our paper:

Despite the panoramas in CVUSA being cropped and having a non-standard aspect ratio, our spherical transform method successfully projects them to a bird's-eye view by completing it to the correct proportion, as shown in Figure 5.

The code for this completion is as follows(we set dty = -20 for CVUSA):

Hp, Wp = img.shape[0], img.shape[1] # pano shape

if dty != 0 or Wp != 2*Hp:

    ty = (Wp/2-Hp)/2 + dty                                                 # completing pano to the correct proportion

    matrix_K = np.array([[1,0,0],[0,1,ty],[0,0,1]])

    img = cv2.warpPerspective(img,matrix_K,(int(Wp),int(Hp+(Wp/2-Hp))))

Typically, cropping involves trimming the content of the vehicles within the field of view. You can also test the bird's-eye view projection of CVUSA panoramic images on our online test platform in Example 3 using the following link.

liguopeng0923 commented 1 year ago

Thanks very much. I have got successfully reasonable results according to your API. Great work!

HaoDot commented 3 hours ago

Hi @liguopeng0923 and @guanfang12. I have set dty = -20 as your suggestion. But the transformation result is still strange like this. I also used cvusa dataset and the resolution of image is 224, 1232. Can you help me to fix it? Thanks for your time. image