TRI-ML / vidar

Other
563 stars 67 forks source link

Inconsistency on PackNet and DepthFormer datasets and evaluation results #42

Closed HoangChuongNguyen closed 1 year ago

HoangChuongNguyen commented 1 year ago

In the paper of PackNet, the reported Abs Rel is 0.162. In the GitHub repository for PackNet (self-supervised), the Abs Rel is reported as 0.173, attributed to the updated DDAD dataset. This repository suggests other methods to compare with the new result, instead of the old one. However, the DepthFormer paper states a result of PackNet's Abs Rel as 0.162, which appears to be an old result.

I have few questions here:

Thank you!

VitorGuizilini-TRI commented 1 year ago

Hi, we apologize for the confusion. The DepthFormer paper should have cited the updated DDAD numbers (with an Abs.Rel. of 0.173). We don't have the splits for the old dataset anymore, so all of our experiments are conducted with the new version (the one that is released). I hope this clarifies things!

HoangChuongNguyen commented 1 year ago

That clears things out. Thank you very much!