Error(s) in loading state_dict for InstructNeRF2NeRFModel

[trltzy]

My command is "ns-train in2n-tiny --data video --load-dir outputs/video/nerfacto/2023-07-11_121450/nerfstudio_models --max-num-iterations 3000 --pipeline.prompt {"give it a cowboy hat"} --pipeline.guidance-scale 7.5 --pipeline.image-guidance-scale 1.5 nerfstudio-data --downscale-factor 4", but I have the error below:

Loading latest Nerfstudio checkpoint from load_dir... Traceback (most recent call last): File "/root/miniconda3/envs/nerfstudio/bin/ns-train", line 8, in sys.exit(entrypoint()) File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 261, in entrypoint main( File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 246, in main launch( File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 189, in launch main_func(local_rank=0, world_size=world_size, config=config) File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 99, in train_loop trainer.setup() File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/engine/trainer.py", line 185, in setup self._load_checkpoint() File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/engine/trainer.py", line 407, in _load_checkpoint self.pipeline.load_pipeline(loaded_state["pipeline"], loaded_state["step"]) File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/pipelines/base_pipeline.py", line 394, in load_pipeline self.load_state_dict(state, strict=True) File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/nerfstudio/pipelines/base_pipeline.py", line 123, in load_state_dict self.model.load_state_dict(model_state, strict=strict) File "/root/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for InstructNeRF2NeRFModel: Missing key(s) in state_dict: "field.mlp_base_grid.hash_table", "field.mlp_base.0.hash_table", "proposal_networks.0.encoding.hash_table", "proposal_networks.0.mlp_base.0.hash_table", "proposal_networks.1.encoding.hash_table", "proposal_networks.1.mlp_base.0.hash_table". Unexpected key(s) in state_dict: "field.direction_encoding.params", "field.position_encoding.params", "field.mlp_base.params", "field.mlp_head.params", "proposal_networks.0.mlp_base.params", "proposal_networks.1.mlp_base.params", "lpips.net.scaling_layer.shift", "lpips.net.scaling_layer.scale".

[ayaanzhaque]

Do you have the newest version of the code installed?

[ayaanzhaque]

I think one issue is that your nerfacto checkpoint is from a month ago. the main nerfacto method is always updating, which is why there is likely some key errors. I recommend retraining the nerfacto model quickly.

[trltzy]

yep, I use a model trained a month ago, I will try a new model, thx for your reply

[trltzy]

thx! The issue get solved as I retrained the nerfactio model.

[ayaanzhaque]

Great!