search

YvanYin / Metric3D

The repo for "Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image" and "Metric3Dv2: A Versatile Monocular Geometric Foundation Model..."
https://jugghm.github.io/Metric3Dv2/
BSD 2-Clause "Simplified" License
1.34k stars 100 forks source link

Got Error decoder_feature_nan!!! and decoder_feature_inf with hubconf.py #121

Open MENG-G opened 3 months ago

MENG-G commented 3 months ago

I am using the hubconf.py from Metric 3D for inference and using the demo images from the code repository. However, I found that the inference results are completely incorrect. Additionally, I’m encountering the following error messages:

decoder_feature_nan!!!
torch.Size([1, 1024, 44, 76])
decoder_feature_inf!!!
torch.Size([1, 1024, 44, 76])
ref_feat_nan!!!
prob_feat_nan!!!
d_nan!!!
norm_nan!!!
abs_rel_err: nan

I have checked the code in mono/model/decode_heads/RAFTDepthNormalDPTDecoder5.py. There are no invalid values in the vit_feature output from the encoder. However, after encoder_features = self.token2feature(vit_features), a lot of invalid values appear. I have tried all the ViT models and encountered this problem with each of them.

Have you come across this issue before?

JUGGHM commented 3 months ago

Have you loaded the pre-trained weights? @ZachL1 Could you please help check the correctness of hubconf models?

ZachL1 commented 3 months ago

I tested it, and everything seems to be working fine. How did you use the sample code? Can you provide more information?