search

FilippoAleotti / mobilePydnet

Pydnet on mobile devices
Apache License 2.0
253 stars 40 forks source link

About evaluation and loss function #43

Closed mstfc closed 2 years ago

mstfc commented 2 years ago

Hi sir,

Thank you for the great project. I would like to improve the model based on the method of your paper. Is it possible to release evaluation script that I can reproduce the evaluation benchmark results on the paper by the pre-trained model? So that I can examine my results to check if I got something wrong while in training process.

I also have another question, did you calculate the multi-scale gradient loss by using output of the upsampling layers? If not, may I know how did you realize the loss calculation?

Thank you!

FilippoAleotti commented 2 years ago

Hi,

I've added the code for KITTI, TUM and NYU v2. The scripts also contain the instructions for each dataset, now you should be able to replicate the numbers reported in the paper. About the loss, yes you have to upsample intermediate depths up to full-res before computing the loss.

mstfc commented 2 years ago

Hi Filippo,

Thanks a lot for the evaluation scripts, it does help me a lot to clear issues. Now I can get similar results (better in Nyuv2, a little worse in KITTI). I didn't sort dataset like you did, so I think it's because of dataset unbalanced.

Again, thanks for your support.