search

tsunghan-wu / Depth-Completion

🍀 Official pytorch implementation of "Indoor Depth Completion with Boundary Consistency and Self-Attention. Huang et al. RLQ@ICCV 2019."
https://arxiv.org/abs/1908.08344
MIT License
180 stars 18 forks source link

MAE and RMSE #8

Open caozidong opened 3 years ago

caozidong commented 3 years ago

I find that MAE and RMSE should use the number of observed pixels in labels as divisor. But you use the number (320x256) for evaluation. Could you explain it? Thank you!

tsunghan-wu commented 3 years ago

Hi,

Yes as you've mentioned, we indeed use the number of observed pixels in evaluation. Our implementation can be shown here. We first calculate the number of unobserved pixels and then calculate # of all pixels - # of unobserved. (where # of all pixels = 320x256) Feel free if you have any further questions. Thanks!

Best Regards, Tsung-Han Wu

caozidong commented 3 years ago

It's kind of you! I still have some questions about it. L1 and RMSE  use ''np.mean'' (line 82 and 83)in your code, which means the divisor is "320x256". The number of observed pixels "N" you mentioned are only used for "delta_*"(line 85-89), instead of L1 and RMSE.  That's my confusion. Thank you!

tsunghan-wu commented 3 years ago

Hi,

Thanks for your great findings! It seems that our implementation might be incorrect. However, I don't think it would affect the comparison results between these depth completion methods. Sincerely sorry to make you confused.

Best Regards, Tsung-Han Wu