Significant degration only changing camera intrinsics

[jingyibo123]

Hi,

Congratulations on your great work and thanks for sharing!

Upon testing on the scene-agnostic model as in README.md, I tried changing only the focal length K[0, 0], K[1, 1] of a test camera, mimicing a new camera intrinsics.

Script for testing is the same as in READM.md

python exp.py --net rnn_vgg16unet3_gruunet4.64.3 --cmd eval --iter last --eval-dsets tat-subseq --eval-scale 0.5

The results are significantly worse with respect to original K. I cannot upload images for security reasons, but here's the code to change the intrinsics

# Changing the focal length of first test image of Truck
Ks = np.load('ibr3d_tat/training/Truck/dense/ibr3d_pw_0.50.bak/Ks.npy')

id=172

print(Ks[id])

Ks[id][0, 0] *= 0.6
Ks[id][1, 1] *= 0.6

np.save('ibr3d_tat/training/Truck/dense/ibr3d_pw_0.50/Ks.npy', Ks)

From the paper the method should be more or less robust to the test-time camera intrinsics/extrinsics, since the features are "attached" to a geometry proxy. Which part of the network could this over-fitting occur?

[jingyibo123]

Turns out it's necessary to update the count_XXX.npy files as well. Or just rerun the create_custom_track