search

isl-org / MiDaS

Code for robust monocular depth estimation described in "Ranftl et. al., Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer, TPAMI 2022"
MIT License
4.25k stars 597 forks source link

Errors during operation for the model swin2t16_256 #236

Open sjanbs opened 10 months ago

sjanbs commented 10 months ago

RuntimeError: Error(s) in loading state_dict for DPTDepthModel: Missing key(s) in state_dict: "pretrained.model.layers.3.downsample.reduction.weight", "pretrained.model.layers.3.downsample.norm.weight", "pretrained.model.layers.3.downsample.norm.bias", "pretrained.model.head.fc.weight", "pretrained.model.head.fc.bias". Unexpected key(s) in state_dict: "pretrained.model.layers.0.downsample.reduction.weight", "pretrained.model.layers.0.downsample.norm.weight", "pretrained.model.layers.0.downsample.norm.bias", "pretrained.model.layers.0.blocks.1.attn_mask", "pretrained.model.layers.1.blocks.1.attn_mask", "pretrained.model.head.weight", "pretrained.model.head.bias".

gene-walt commented 9 months ago

You have both "missing keys" and "unexpected keys", I was getting the just unexpected keys trying to use the _dpt_beit_large512 model.

What do you see if you edit base_model.py to print all the keys? Are those keys missing?

for key in parameters:
    print(key)

Because I had unexpected keys, I simply deleted all the bad keys before calling load_state_dict. But then I ran into another issue: https://github.com/isl-org/ZoeDepth/issues/26

wgqt1zl commented 8 months ago

You have both "missing keys" and "unexpected keys", I was getting the just unexpected keys trying to use the _dpt_beit_large512 model.

What do you see if you edit base_model.py to print all the keys? Are those keys missing?

for key in parameters:
    print(key)

Because I had unexpected keys, I simply deleted all the bad keys before calling load_state_dict. But then I ran into another issue: isl-org/ZoeDepth#26

hello have you solved the problem?

gene-walt commented 8 months ago

You have both "missing keys" and "unexpected keys", I was getting the just unexpected keys trying to use the _dpt_beit_large512 model. What do you see if you edit base_model.py to print all the keys? Are those keys missing?

for key in parameters:
    print(key)

Because I had unexpected keys, I simply deleted all the bad keys before calling load_state_dict. But then I ran into another issue: isl-org/ZoeDepth#26

hello have you solved the problem?

I have MiDaS working with dpt 512.

I haven’t made any pull request since I’m guessing my issue is related to my personal Windows environment. I think most ppl are running on Mac? After manually deleting keys to get around the “unexpected” key issue, I also had to change my “timm” module version inside of the environment file (https://github.com/isl-org/ZoeDepth/issues/26)

This won’t help GitHub issue though as the error says “missing keys”.

Waving-Flag commented 6 months ago

RuntimeError: Error(s) in loading state_dict for DPTDepthModel: Missing key(s) in state_dict: "pretrained.model.layers.3.downsample.reduction.weight", "pretrained.model.layers.3.downsample.norm.weight", "pretrained.model.layers.3.downsample.norm.bias", "pretrained.model.head.fc.weight", "pretrained.model.head.fc.bias". Unexpected key(s) in state_dict: "pretrained.model.layers.0.downsample.reduction.weight", "pretrained.model.layers.0.downsample.norm.weight", "pretrained.model.layers.0.downsample.norm.bias", "pretrained.model.layers.0.blocks.1.attn_mask", "pretrained.model.layers.1.blocks.1.attn_mask", "pretrained.model.head.weight", "pretrained.model.head.bias".

It seems that the versions of your imutils and timm Python libraries might be too high. You can try lowering their versions according to the specifications in the environment.yaml file. That's how I resolved a similar issue.