Provided checkpoint incompatible with provided dataset

[kochsebastian]

I have one a small problem with your provided checkpoints.

I downloaded the 3DSSG dataset from here: https://campar.in.tum.de/public_datasets/3DSSG/3DSSG.zip including the classes.txt which has 528 classes. I downloaded the checkpoints from here: https://drive.google.com/drive/folders/1RbNUpHLQiY1zskd0buAgvl-yFwnD9N8M?usp=sharing

When I try to evaluate the model using:

python scripts/evaluate_vaegan.py --dataset_3RScan ../3RScan_v2/data/ --exp ./experiments/final_checkpoints/shared --with_points False --with_feats True --epoch 100 --path2atlas ./experiments/atlasnet/model_70.pth --evaluate_diversity False

I get the following error:

RuntimeError: Error(s) in loading state_dict for Sg2ScVAEModel:
        size mismatch for obj_embeddings_ec_box.weight: copying a param with shape torch.Size([162, 128]) from checkpoint, the shape in current model is torch.Size([530, 128]).
        size mismatch for obj_embeddings_ec_shape.weight: copying a param with shape torch.Size([162, 128]) from checkpoint, the shape in current model is torch.Size([530, 128]).
        size mismatch for pred_embeddings_ec_box.weight: copying a param with shape torch.Size([27, 256]) from checkpoint, the shape in current model is torch.Size([41, 256]).
        size mismatch for pred_embeddings_ec_shape.weight: copying a param with shape torch.Size([27, 256]) from checkpoint, the shape in current model is torch.Size([41, 256]).
        size mismatch for obj_embeddings_dc_box.weight: copying a param with shape torch.Size([162, 256]) from checkpoint, the shape in current model is torch.Size([530, 256]).
        size mismatch for obj_embeddings_dc_man.weight: copying a param with shape torch.Size([162, 256]) from checkpoint, the shape in current model is torch.Size([530, 256]).
        size mismatch for obj_embeddings_dc_shape.weight: copying a param with shape torch.Size([162, 256]) from checkpoint, the shape in current model is torch.Size([530, 256]).
        size mismatch for pred_embeddings_dc_box.weight: copying a param with shape torch.Size([27, 512]) from checkpoint, the shape in current model is torch.Size([41, 512]).
        size mismatch for pred_embeddings_dc_shape.weight: copying a param with shape torch.Size([27, 512]) from checkpoint, the shape in current model is torch.Size([41, 512]).
        size mismatch for pred_embeddings_dc.weight: copying a param with shape torch.Size([27, 256]) from checkpoint, the shape in current model is torch.Size([41, 256]).
        size mismatch for pred_embeddings_man_dc.weight: copying a param with shape torch.Size([27, 768]) from checkpoint, the shape in current model is torch.Size([41, 768]).

I think this error is generated because self.obj_embeddings_ec_box = nn.Embedding(num_objs + 1, obj_embedding_dim) in Sg2ScVAEModel in VAEGAN_SHARED.py takes the number of classes as the number of input features. And the number of classes is defined by the length of vocab['object_idx_to_name'] which is loaded from the classes.txt which is 528 classes + the first row scene. This similar for the relationships and self.pred_embeddings_ec_box = nn.Embedding(num_preds, 2 * embedding_dim).

Seams like you trained only with 161 classes and 27 relationships. Could you provide me with the correct classes.txt file and relationships.json to replicate your results?

[he-dhamo]

Hi there, thanks for pointing this out. The classes.txt and relationships.txt files for the subset of classes that we train with, can be found here: https://campar.in.tum.de/public_datasets/3DSSG/3DSSG_subset.zip

[kochsebastian]

Thank you for the quick response 🙂