search

RaymondWang987 / NVDS

ICCV 2023 "Neural Video Depth Stabilizer" (NVDS) & TPAMI 2024 "NVDS+: Towards Efficient and Versatile Neural Stabilizer for Video Depth Estimation" (NVDS+)
MIT License
487 stars 24 forks source link

Reproducing the results on the NVUDv2 dataset #2

Closed zuoym15 closed 1 year ago

zuoym15 commented 1 year ago

Thanks for the amazing work and releasing the code! Are you planning to release the checkpoints and/or test script for the NYUDv2 dataset, so that I can reproduce the numbers in the paper?

RaymondWang987 commented 1 year ago

Thanks for the amazing work and releasing the code! Are you planning to release the checkpoints and/or test script for the NYUDv2 dataset, so that I can reproduce the numbers in the paper?

Hi! Thanks for your attention to our work. As scheduled in our TODO list, we will release the NVDS checkpoint (finetuned on NYUDV2) with MiDaS and DPT as different depth predictors, along with the evaluation script for reproducing the depth metrics in several weeks.

RaymondWang987 commented 1 year ago

Thanks for the amazing work and releasing the code! Are you planning to release the checkpoints and/or test script for the NYUDv2 dataset, so that I can reproduce the numbers in the paper?

Hi! We have released the evaluation code and checkpoint of NYUDV2-finetuned NVDS. You can run the code as instructed in the readme file and reproduce the depth metrics on NYUDV2 as our paper.

cvtype commented 1 year ago

Hi @RaymondWang987 Thanks for sharing the finetuned model on NYUDV2, but seems the stabilization ability is not as good as the original model on VDW.

RaymondWang987 commented 1 year ago

Hi @RaymondWang987 Thanks for sharing the finetuned model on NYUDV2, but seems the stabilization ability is not as good as the original model on VDW.

NYUDV2 exists some problems for training consistent video depth models：(1)NYUDV2 contains some obvious flickers in the ground-truth，for example some reflection areas and the holes completed by its pre-processing toolkits. (2) The image quality of NYUDV2 is low. You can see spotted noise when the frames are zoomed in (this is also the main reason for the requests of finetuning NVDS on NYUDV2 or KITTI, the RGB quality is the main gap since VDW dataset also contains diverse indoor scenes). Thus, the stabilization ability on NYUDV2 could be not as good as the original model on VDW. But we also achieves obvious improvements than the initial depth predictors. We only conduct the experiments on NYUDV2 for comparisons to prove the effectiveness of NVDS and VDW dataset.

cvtype commented 1 year ago

@RaymondWang987 Thanks very much for the detailed explanation.