Closed hj-jung-01 closed 4 months ago
Make sure you're using the right combination of model and checkpoint. The original version of the model and the improved camera-ready version of the model are slightly different, so the checkpoints aren't compatible. I would recommend pulling the latest commit and using these checkpoints.
Hi, I'm trying to run evaluation on re10k subset following your documentation. But loading pretrained model fails.
Error executing job with overrides: ['+experiment=re10k', 'mode=test', 'dataset/view_sampler=evaluation', 'dataset.view_sampler.index_path=re10k_subset/test/index.json', 'checkpointing.load=checkpoints/re10k.ckpt'] Traceback (most recent call last): File "/nasspace/samsung/pixelsplat/src/main.py", line 130, in train trainer.test( File "/opt/conda/lib/python3.10/site-packages/lightning/pytorch/trainer/trainer.py", line 754, in test return call._call_and_handle_interrupt( File "/opt/conda/lib/python3.10/site-packages/lightning/pytorch/trainer/call.py", line 44, in _call_and_handle_interrupt return trainer_fn(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/lightning/pytorch/trainer/trainer.py", line 794, in _test_impl results = self._run(model, ckpt_path=ckpt_path) File "/opt/conda/lib/python3.10/site-packages/lightning/pytorch/trainer/trainer.py", line 956, in _run self._checkpoint_connector._restore_modules_and_callbacks(ckpt_path) File "/opt/conda/lib/python3.10/site-packages/lightning/pytorch/trainer/connectors/checkpoint_connector.py", line 398, in _restore_modules_and_callbacks self.restore_model() File "/opt/conda/lib/python3.10/site-packages/lightning/pytorch/trainer/connectors/checkpoint_connector.py", line 275, in restore_model self.trainer.strategy.load_model_state_dict( File "/opt/conda/lib/python3.10/site-packages/lightning/pytorch/strategies/strategy.py", line 372, in load_model_state_dict self.lightning_module.load_state_dict(checkpoint["state_dict"], strict=strict) File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for ModelWrapper: Unexpected key(s) in state_dict: "encoder.epipolar_transformer.transformer.layers.0.1.fn.layers.0.weight", "encoder.epipolar_transformer.transformer.layers.0.1.fn.layers.0.bias", "encoder.epipolar_transformer.transformer.layers.0.1.fn.layers.3.weight", "encoder.epipolar_transformer.transformer.layers.0.1.fn.layers.3.bias", "encoder.epipolar_transformer.transformer.layers.1.1.fn.layers.0.weight", "encoder.epipolar_transformer.transformer.layers.1.1.fn.layers.0.bias", "encoder.epipolar_transformer.transformer.layers.1.1.fn.layers.3.weight", "encoder.epipolar_transformer.transformer.layers.1.1.fn.layers.3.bias".
The excuted command is
python3 -m src.main +experiment=re10k mode=test dataset/view_sampler=evaluation dataset.view_sampler.index_path=re10k_subset/test/index.json checkpointing.load=checkpoints/re10k.ckpt
I have traced the implementation of epipolar transformer, there is no variable named "layers" or use of "ModuleList" in attension.py. So there is completely missing 8 layers.
I think the pretrained model's network architecture and the implementation are not matched. Can you check out this error?