I trained this model with my dataset. It is ok while training, but there are some strange problem when I want to use the saved checkpoints to inference.
The saved pt files have strange dict_key: dict_keys(['module', 'buffer_names', 'optimizer', 'param_shapes', 'lr_scheduler', 'data_sampler', 'random_ltd', 'sparse_tensor_module_names', 'skipped_steps', 'global_steps', 'global_samples', 'dp_world_size', 'mp_world_size', 'ds_config', 'ds_version', 'epoch', 'global_step', 'pytorch-lightning_version']) which is different from the model.ckpt
no 'state_dict' means the model weights can not load correctly while inferencing.
Could you tell me how to save the trained model and load it correctly, pls?
HadiZayer
commented
11 months ago
I trained this model with my dataset. It is ok while training, but there are some strange problem when I want to use the saved checkpoints to inference.
The saved pt files have strange dict_key: dict_keys(['module', 'buffer_names', 'optimizer', 'param_shapes', 'lr_scheduler', 'data_sampler', 'random_ltd', 'sparse_tensor_module_names', 'skipped_steps', 'global_steps', 'global_samples', 'dp_world_size', 'mp_world_size', 'ds_config', 'ds_version', 'epoch', 'global_step', 'pytorch-lightning_version']) which is different from the model.ckpt
no 'state_dict' means the model weights can not load correctly while inferencing.
Could you tell me how to save the trained model and load it correctly, pls?