Test mesh is "ballooned" / where are UV mappings?

[atonalfreerider]

I ran train.py and test.py on the Taichi data, and I received output .plys in the Test folders. The plys are "ballooned" ie they are swollen up larger than the figures in the video. Is this because the algorithm did not finish "shrinking" them into the masks? Is something supposed to output to the Stage folders as well?

Also, are UV textures and mappings supposed to be output somewhere?

Thank you. Also I understand why the project is named "Multi-Ply" now :)

[jzr99]

Hi,

How many epochs do you train? It seems like the model is not fully converged. Also, it usually takes 48 hours of training for two people.

As for the UV textures, our method doesn't support outputting textured meshes directly. But one way to try is rendering multiple images for a certain frame and re-projecting the RGB values to the mesh of the current frame. Or you can try to directly query the radiance network in canonical space for each point in the output meshes to get the vertex color.

[atonalfreerider]

Thank you. Not very many epochs. Which configuration parameter controls this?

train:
    dataset: Hi4D
    subject: "taichi01"
    start_frame: 0
    end_frame: 75
    batch_size: 1
    drop_last: False
    shuffle: True
    worker: 8
    num_person: 2
    data_dir : taichi01_vitpose_openpose/
    num_sample : 512
    using_SAM: True
    ratio_uncertain: 0.5

Also, what are the "stage" folders used for? The models only output to the "test" folders.

Thank you for the recommendation for the UV textures. I'll respond back here if I make any more progress with this

[jzr99]

Change here to set the max training epoch. Or you can manually stop training if the quality of the normal map in the normal folder converged.

Stage folder is used to store some intermediate result during training. E.g. the updated SAM mask during training.