Result seems not right

[zhouzhenghong-gt]

Hello, thanks for your great work! I ran the barf code on the chair of the blender dataset, but the validation result doesn't seem right on tensorboard, is this normal? Train:

Test:

[chenhsuanlin]

Hi @zhouzhenghong-gt, thanks for point this out. You're right that the current validation error is more or less flawed. In the current codebase, the validation error is computed directly between the rendered image at the "given" val pose and the ground-truth image. However, since the entire (reference) coordinate system could be changing, the actual validation rendered image may not be aligned with the ground truth, thus the non-decreasing error.

A better way would indeed be to pre-align the poses with Procrustes analysis (as done in eval mode, see the "evaluation criteria" paragraph in the paper), and I can push an update in the coming days to fix this. However, further test-time optimization would be too computationally prohibitive just for the sake of validation. So even with the upcoming fix, the val loss/PSNR would still not be a perfect metric and should be taken with a grain of salt.

Hope this clarifies! Leaving this issue open for now until the fix is updated.

[zhouzhenghong-gt]

Thanks for your reply! It helps a lot.

By the way, I am curious if you have compared the results of nerf--?

[chenhsuanlin]

I have not unfortunately, would be an interesting comparison though! It should be fairly easy to incorporate NeRF-- into this BARF codebase, essentially (a) additional intrinsics optimization and (b) no coarse to fine in pose optimization.

[chenhsuanlin]

Hi @zhouzhenghong-gt, sorry for the delay -- I have just pushed the fix (see 803291b). The code now validates on the pre-aligned poses with Procrustes analysis, and you should see the validation loss/PSNR and rendered images as below. Hope this helps!

.