search

RaymondWang987 / FMNet

The official project of ACM MM 2022 paper "Less is More: Consistent Video Depth Estimation with Masked Frames Modeling".
Apache License 2.0
30 stars 2 forks source link

Help Improving Predictions #6

Open joreeves opened 1 year ago

joreeves commented 1 year ago

Hi, thank you for putting this code together.

I am testing it out and trying to understand if I am doing something wrong. Here are the results I've gotten which is quite flickery compared to the original example. Any pointers on what parameters need to be adjusted would be helpful, thank you.

I've left small_seq_len=2

https://github.com/RaymondWang987/FMNet/assets/82180221/7c442522-efd2-4f47-ad87-5f3db885f573

FMNet_Test_Frames.zip

RaymondWang987 commented 11 months ago

Hi！Sorry for the late reply.

Your test sequence is for outdoor driving scenes. However, the checkpoint contained in my FMNet repo is only trained on indoor NYUDV2 dataset. There is a large domain gap between the training data and your test sequence, which leads to obvious performance degradation. In this case, you should train the model on outdoor driving datasets and the results will be improved. Besides, you can also try on our work "Neural Video Depth Stabilizer"(NVDS) accepted by ICCV2023 (https://github.com/RaymondWang987/NVDS). NVDS is trained on large scale natural-scene dataset and can deal with many in-the-wild videos.