Train on a new model: mesh_vertex_embeddings are all zeros in my model

[Haozong-Zeng]

I am using DensePose CSE for capturing poses from videos and guiding a mesh model. My goal is to use my own mesh and the corresponding dataset to train a DensePose CSE model.

I formated the dataset & annotation correctly, and successfully registered the mesh by providing the vertices, geometric distances, symmetry, etc. information. The only thing I missed is the INIT_FILE which contains 'eigenvalues', 'features', & 'baricel_area'. My trained model seems to work well on providing the

E (tensor [1, D, H, W]): D-dimensional embedding vectors for every point of the default-sized box

&

S (tensor [1, 2, H, W]): 2-dimensional segmentation mask for every point of the default-sized box

However, when I tried to find the closest vertices, I noticed that the

mesh_vertex_embeddings (tensor [N, D]): vertex embeddings for a chosen mesh N is the number of vertices in the mesh, D is feature dimensionality

is a all-zeros tensor and thus I could not calculate the squared_euclidean_distance_matrix.

When training the model, I received the following warning:

Some model parameters or buffers are not found in the checkpoint: roi_heads.embedder.embedder_my_mesh.{embeddings, features}

1. Full changes you made: _BASE_: "Base-DensePose-RCNN-FPN.yaml" MODEL: WEIGHTS: "https://dl.fbaipublicfiles.com/densepose/cse/a_checkpoint.pkl" RESNETS: DEPTH: 50 ROI_HEADS: NUM_CLASSES: 1 ROI_DENSEPOSE_HEAD: NAME: "DensePoseV1ConvXHead" COARSE_SEGM_TRAINED_BY_MASKS: True CSE: EMBED_LOSS_NAME: "SoftEmbeddingLoss" EMBEDDING_DIST_GAUSS_SIGMA: 0.1 GEODESIC_DIST_GAUSS_SIGMA: 0.1 EMBEDDERS: "my_mesh": TYPE: vertex_feature NUM_VERTICES: n FEATURE_DIM: 256 FEATURES_TRAINABLE: True (also try False) IS_TRAINABLE: True INIT_FILE: ""

2. What exact command you run: The train_net.py script in the tutorial

[github-actions[bot]]

You've chosen to report an unexpected problem or bug. Unless you already know the root cause of it, please include details about it by filling the issue template. The following information is missing: "Instructions To Reproduce the Issue and Full Logs"; "Your Environment";

[github-actions[bot]]

Requested information was not provided in 7 days, so we're closing this issue.

Please open new issue if information becomes available. Otherwise, use github discussions for free-form discussions.