search

EndyWon / Texture-Reformer

Official Pytorch code for "Texture Reformer: Towards Fast and Universal Interactive Texture Transfer" (AAAI 2022)
MIT License
54 stars 5 forks source link

Concat Weight, Pattern and synthesis #3

Closed pjuchs closed 2 years ago

pjuchs commented 2 years ago

Hello again, finally I am still trying to do something with your Texture-Reformer, but what I am trying to do goes (past) the limits explained in the article, in term of scaling/rotating elements. But I am still asking if you think it is possible use this method to generate texture by indicating (with the semantic) only one part of the style image and generate with this part a bigger texture. By the way the goal is to use a BaselFaceModel face texture to obtain a full body texture.

I have done different things, and played with the concat weight awareness, for example : style: face3d6 style_sem: 512face3d_sem content_sem: b9_sem result with a patch size of 15 : female-i17-p15

There the result isn't perfect but it's not too slow and it's good enough just for the skin. We can try to use the style image for the face even if again there are some readjustment problems, but we have no hair.

So I wanted to print the hair too on the same image or a different one : b10_sem hair_sem

But the result isn't there : b10_512female_i17p15_w=50 hair_512female_i17p15_w=50

It might be hard to understand what I ask there. So what i want to know is if you think it is possible to generate a texture with this method, using a small input from the style image. Either like we tried with a pattern like showed in t previous image or with a different application. I also wanted to know if there is a way to play with the concat weight of stage I and II separately like showed in the end of the article.

I am asking you that even thought the pattern application doesn't to seem very correct : Since when we don't have the pattern on the face we still have that: face512sanspattern b9_sem b9_512female_0p

And this is with the pattern, there is a difference but not as big as i would think : 512face3d_sem b9_sem b9_512female_1p

The black dot have still some use cause if there is only green it does that : face512sanspattern green_sem green_512female_0p

Well I hope you can help me again, and I am sorry for the chaos of my questions and explanations.

EndyWon commented 2 years ago

Thanks for the questions. I tried to understand your questions. My suggestions are as follows: (1) Removing the global alignment stage I, and using the local stage II only; (2) Cropping your wanted texture region from the style image, and using the cropped image as the new style image (with proper size); (3) If you want to play with the concat weight of stage I and II separately, just modify the codes in Line123-158 and Line199/202 in transfer.py.

Hope these could help you, good luck!