search

miccunifi / ladi-vton

[ACM MM 2023] - LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-On
Other
412 stars 56 forks source link

[Colab Guide] Drive Link for Quick Inference on custom data with LADI-VTON using Colab #25

Open kira1433 opened 1 year ago

kira1433 commented 1 year ago

I have made a Colab preprocessing pipeline for Ladi-VTON which can run inference on custom data using the DressCode model.

Here is the my drive Link https://drive.google.com/drive/folders/19XL0kvTw6SoCCAOJY9FgvuQJ9M_JAZHt?usp=sharing You will need to make a copy of my drive in your google drive with the same name first and use GPU on Colab

I have made pre-processing usable for the DressCode dataset. Keep your input images in /images folder and write the test pairs properly. Then after running ladi-vton_DressCode.ipynb input folder for inference will automatically be generated.

By running inference on custom data, ladi-vton messes up the faces. So I have made a Refinement Notebook using Google Mediapipe just for this purpose, The intermediate results after inference are in results folder, and final results after refinement will be in final folder.

I have used this exact drive to generate some results and it mostly works. There are some problems with few specific garments. Thinking to shift the drive into a Github repo after sometime. If you have any doubts/suggestions you can post.

sunnytomy commented 1 year ago

Thanks a million for your sharing. As I went through result images, I still can tell details difference between the original garment and the one dressed on human. The difference becomes more outstanding when texture graph is more complex. For pure color garments, the results are pretty good. Any suggestion to improve texture or design graph replication?

kira1433 commented 1 year ago

Thanks a million for your sharing. As I went through result images, I still can tell details difference between the original garment and the one dressed on human. The difference becomes more outstanding when texture graph is more complex. For pure color garments, the results are pretty good. Any suggestion to improve texture or design graph replication?

Yeah, I have noticed the same. To be fair, this model doesn't provide proper sizing or fine details ,but I think that's a limitation of Diffusion models right now. The only great point is that it works

I recommend you to check out papers like HR-Viton, if you need to work with complex clothing.

sunnytomy commented 1 year ago

Thanks a million for your sharing. As I went through result images, I still can tell details difference between the original garment and the one dressed on human. The difference becomes more outstanding when texture graph is more complex. For pure color garments, the results are pretty good. Any suggestion to improve texture or design graph replication?

Yeah, I have noticed the same. To be fair, this model doesn't provide proper sizing or fine details ,but I think that's a limitation of Diffusion models right now. The only great point is that it works

I recommend you to check out papers like HR-Viton, if you need to work with complex clothing.

Google unveiled its diffusion based tryon program last month and the result is quite impressive regarding garment details preservation. Therefore i think diffusion should be the right way to go even though it still has these or other problems. Actually if you train lora model for each garment, you can gain good results too when not considering identical texture replicated. I have tested HR-VITON, it presented some good results, but most of them are still far form perfect, it shares some common cons with other VITON.

nazapip commented 1 year ago

Thanks a lot friend!!.. This is awesome.. will start working on it now.. Can you please elaborate more on this refinement notebook.. i had been facing this issue from long time about the face change problem and you brought up the solution, i want to know about its code you wrote, is there any reference from where you got this? or any guide which i can read about it more and yes.. how can i integrate this with the final results directly if an input of cloth and image is given and i want the final image directly without changing the face

RishiGitH commented 1 year ago

@kira1433 I have been trying to solve this problem for a while . What would u recommend is the Viton-HD version better or the dresscode version for this project . The dresscode version you shared had issues with new dress . I tried it on this dress and a new image and this was my result . Are there any other opensource model that work better . I need this for my personal project

003_final (1) 003

kira1433 commented 1 year ago

Thanks a lot friend!!.. This is awesome.. will start working on it now.. Can you please elaborate more on this refinement notebook.. i had been facing this issue from long time about the face change problem and you brought up the solution, i want to know about its code you wrote, is there any reference from where you got this? or any guide which i can read about it more and yes.. how can i integrate this with the final results directly if an input of cloth and image is given and i want the final image directly without changing the face

No I wasn’t able to get good results without using the refinement I mentioned.

kira1433 commented 1 year ago

@kira1433 I have been trying to solve this problem for a while . What would u recommend is the Viton-HD version better or the dresscode version for this project . The dresscode version you shared had issues with new dress . I tried it on this dress and a new image and this was my result . Are there any other opensource model that work better . I need this for my personal project

003_final (1) 003

I think probably the Viton-HD version is better for just top-wear. I think this is one of best models now and I don’t think you will find much improvement elsewhere. But you can experiment.

TrongDuyNguyen0611 commented 11 months ago

@kira1433 I have been trying to solve this problem for a while . What would u recommend is the Viton-HD version better or the dresscode version for this project . The dresscode version you shared had issues with new dress . I tried it on this dress and a new image and this was my result . Are there any other opensource model that work better . I need this for my personal project 003_final (1) 003

I think probably the Viton-HD version is better for just top-wear. I think this is one of best models now and I don’t think you will find much improvement elsewhere. But you can experiment.

What do you think about HR-VITON model?

ShroukMansour commented 7 months ago

hello, can i use this notebook for vton-hd dataset? @kira1433

PraNavKumAr01 commented 6 months ago

@kira1433 Hey ive been trying to use your code for custom dataset, but its really hard to understand the organization, Lets say i have a model image, and an image of another model wearing the cloth that i want on the first model, i just need to run inference on 1 image, what process should i follow?

PraNavKumAr01 commented 6 months ago

@kira1433 I have figured out the rest, i just have a doubt where do i need to put my input model image, and the input cloth image?As in which folder to start with? Can anybody else help?