About the idea of pseudo training pairs

[selfitcamera]

First of all, congratulations on your success! The logic of the paper looks very promising and the renderings are also very good ! It is a great work !

The concern about use pseudo training pairs to train a model

Using fake data created by other models to train models is an effective method in many fields. The field of target detection and recognition usually has similar ideas such as Pseudo-Labelling and semi-supervised learning.

However, when it comes to virtual fitting, I have a little doubt that compared with the real groundtruth, the effects of most virtual fitting models are very limited. The effects are very good in the presentation pictures of the paper, but when they are actually tested in wild data, various problems often occur.

The image classification field is different. In the target detection field, the reason why pseudo-label training can work is that target classification is a simple task. The model for making pseudo-labels has a high accuracy rate. After manual screening, pseudo-labels are actually high-quality data. Therefore, the model trained based on pseudo-labels can often break through the original annotation model for making pseudo-labels.

In the field of VTON, maybe the upper limit of the model trained based on this pseudo-label data is determined by the annotation model for making pseudo-labels. In theory, its virtual fitting generation effect will not exceed the annotation VTON model you used?

My guess I understand that the core of the paper is to implement a Mask-Free virtual fitting method, so the problem mentioned above is another aspect, and it is not the core of this method.

Final Overall, it is a pioneering work to realize Mask-Free through pseudo-labeling！ Congratulations again on your work！ I look forward to the early release of the hugging face demo！ I believe everyone can't wait to test it.

[Xiphin]

Looking forward to it.

[little-misfit]

Thank you very much for your recognition, and your work is also very interesting! Next, I'll try to answer your question. I think your question can be summarized as: "Does the quality of synthetic characters affect the mask-free model ?" The answer is: Absolutely. Existing methods can generate sufficiently realistic characters in simple in-shop scenarios, which provides a basis for creating our synthetic data. We then enhance the diversity of the data through data augmentation. It is also important to note that we use the pseudo person as input rather than ground truth, so the performance limitations of existing methods do not constrain our method. In fact, the methods section of the article covers everything; you can easily reproduce this process.~///(^v^)\\~

[selfitcamera]

Cool 👍️