creating the segmentation json file

[msverma101]

Could you please tell me how did you create these json files for segmenation of each garment. i am trying to use the 3d data generated with cloth from ICON smpl to use as an input to your pipeline to generate garment patterns and in the end produce a garment itself. could you please eloborate on the dataset processing so i can create one for myself

[maria-korosteleva]

If you ask about the sewing pattern .json files, the data we use to train NeuralTailor supplies the ground truth patterns for each garment 3D model. The dataset was created in our previous project, and both the data and the generation pipeline are available: https://zenodo.org/record/5267549 https://github.com/maria-korosteleva/Garment-Pattern-Generator

The data generation process is described in detail in the corresponding paper. In short, patterns are sampled first and then used to create a garment mesh and drape it on top of a body model.

Does this answer your question?

[msverma101]

partly i am trying to combine the output of ICON model here is the link for that. https://github.com/YuliangXiu/ICON this outputs full body volumetric view and also extracted garment there are some errors in it. what i want to do is once the garments are extracted in volumetric form. i want to use it as an input to your pretrained model and predict the garment seams from which view how it would look like in 3d after that.

1. is it possible to use the pretrained model and predict unseen data for example using this command here: python nn/evaluation_scripts/on_test_set.py -sh models/att/att.yaml -st models/att/stitch_model.yaml --unseen --predict

2. since my objective is not to create a new dataset rather just make prediction of seams for a given garment which does not belong to any of the dataset that you trained it on. what are the chances that it would predict well. specially since the test data is going to be a bit noisy.

[msverma101]

mind you this is also in form of OBJ but not exactly point clouds i saw this yaml files that mention the test dataset. if i just replace or add new ones like mine above. what else would i need other that the obj itself to predict the seams

[maria-korosteleva]

1) To use the model on your meshes, you can use this script: predict_per_example.py. It will automatically sample the point cloud from the mesh to match the input NeuralTailor is trained with.

2) As with any data-based method, the quality of predictions will deteriorate if your examples are from from the NeuralTailor data distribution. As discussed in the paper, the dataset contains only V-neck collars, so a circle neck like in the example you have, won't be restored. Same with the pose: NeuralTailor is only trained on garments from bodies in T-pose, so poses with hands down will not work. This is, unfortunately, the limitation in the SOTA of datasets with sewing patterns at the moment. I would encourage you to try and see what works for your outputs, of course.

3) Please note that the stitch prediction module is part of the prediction of sewing patterns -- it tells which edges of the pattern panels should be connected. It's not a mesh segmentation

[msverma101]

so do i have to give with the point cloud or mesh of the garment the sewing patter as well for it to predict the edges of the pattern that will be connected? or does the model also predicts the stick patterns as well from the garment mesh

[maria-korosteleva]

The NeuralTailor model predicts full sewing pattern information: the shape of the fabric panels on the first step, and the stitching between those panels on the second step. Sewing pattern recovery is the main contribution of our paper =)