VITA-Group / GNT

[ICLR 2023] "Is Attention All NeRF Needs?" by Mukund Varma T*, Peihao Wang* , Xuxi Chen, Tianlong Chen, Subhashini Venugopalan, Zhangyang Wang
https://vita-group.github.io/GNT
MIT License
338 stars 24 forks source link

keys mismatch when loading pre-trained models #4

Closed CCOSerika closed 2 years ago

CCOSerika commented 2 years ago

Hi, author. I met an error when loading the pre-trained models. The details are as follows.

outputs will be saved to ./out/gnt_lego
loading ['lego'] for val
Traceback (most recent call last):
  File "eval.py", line 236, in <module>
    eval(args)
  File "/home/vt/anaconda3/envs/gnt/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "eval.py", line 82, in eval
    model = GNTModel(
  File "/media/vt/work/NYXcode/GNT_new/gnt/model.py", line 77, in __init__
    self.start_step = self.load_from_ckpt(
  File "/media/vt/work/NYXcode/GNT_new/gnt/model.py", line 166, in load_from_ckpt
    self.load_model(fpath, load_opt, load_scheduler)
  File "/media/vt/work/NYXcode/GNT_new/gnt/model.py", line 136, in load_model
    self.net_coarse.load_state_dict(to_load["net_coarse"])
  File "/home/vt/anaconda3/envs/gnt/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1482, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for GNT:
    Missing key(s) in state_dict: "view_trans.0.attn_norm.weight", "view_trans.0.attn_norm.bias", ...
    Unexpected key(s) in state_dict: "view_selftrans.0.attn_norm.weight", "view_selftrans.0.attn_norm.bias", ...

I try to modify the mismatching keys, but the correspondence much later is barely perceptible. Do you have any solutions? Thanks in advance.

MukundVarmaT commented 2 years ago

Apologies for the delayed response. For the sake of readability, I had modified one of the layer names in the released code, hence the key mismatch. I have corrected the keys and reuploaded the checkpoints. Please do let me know if the checkpoints work as expected.

CCOSerika commented 2 years ago

Great appreciation for ur kindness. I have tested the ckpts on several datasets including my own indoor scene. The result is amazing and even exceeds my expectation! Congratulations on such an excellent work and ur generously releasing the code.