Should the image input to the neural network be dedistorted?

[vtasStu]

Thank you for your great work. I have a few questions.

1. Should the image be distorted?
2. The code inverse_depth = canonical_inverse_depth * (W / f_px) is a bit confused me. The canonical_inverse_depthcorresponds to the resized image, while the W / f_px corresponds to the original image.

[xiaodongww]

Hi，I also have the same quesiton. By the way, intrinsics usuall have four parameters (fx, fx, cx, cy), is only f_px enough to produce metric depth? do you have any idea? @vtasStu

[JVPC0D3R]

@xiaodongww I used both the fx parameter and a mean between fx and fy. None of them gave me a proper depth map for my custom images.

# horizontal focal length
f_px= torch.tensor(fx)
image = transform(frame)

# mean focal length
f_px= torch.tensor((fx + fy) /2)
image = transform(frame)

After processing the image and getting the metric depth map I ended up with the map below. I tested the same code with several samples and I only got shapes of the furthest elements represented in the map.

[JVPC0D3R]

The depth map is correct, the outputs go really far away in this example (up to 10km). In order to visualize the output in a shorter range you can use depth = depth.clip(0,200)

Regarding the original question, I recomend undistort your image if possible.