search

google / style-aligned

Official code for "Style Aligned Image Generation via Shared Attention"
Apache License 2.0
1.19k stars 86 forks source link

style transfer from an existing reference image? #6

Closed cocktailpeanut closed 8 months ago

cocktailpeanut commented 8 months ago

IMO the highlight of this tech is being able to transfer style from an existing image, and the paper and the homepage discuss this point primarily, which was the reason why I got really excited.

Unfortunately, the repo doesn't include a demo for doing this. I've tried all the other demos but I really think being able to use a reference image for style will be 1000 times more impressive than the existing ones, because while the tech itself is really powerful, the existing demos do not fully demonstrate the power.

Any plans to share at least a minimal demo for using an input reference image for style transfer?

baleksey commented 8 months ago

+1. I think "style transfer" probably is the most valuable result for this technique from the practical point of view. So devs, could you please add the notebook for that?

demiguelm commented 8 months ago

++1. What attracted my attention the most was the possibility of using a really mind-blowing style transfer.

demiguelm commented 8 months ago

Already updated and it works, but EXTREMELY slow. 30.05s/it in 3080ti

cocktailpeanut commented 8 months ago

I got it to work faster by using 512x512 instead of 1024x1024. The 1024x1024 option takes up way too much memory.

With the 512x512, it even works very smooth on M1 macs. If you want to learn more or check out the code, check this out https://x.com/cocktailpeanut/status/1733139983819554858?s=20

And thank you maintainer for making the example code available!

demiguelm commented 8 months ago

I got it to work faster by using 512x512 instead of 1024x1024. The 1024x1024 option takes up way too much memory.

With the 512x512, it even works very smooth on M1 macs. If you want to learn more or check out the code, check this out https://x.com/cocktailpeanut/status/1733139983819554858?s=20

And thank you maintainer for making the example code available!

but 1024 is native for SDXL...

cocktailpeanut commented 8 months ago

Right, so i wasn't sure if it was going to work, but I can confirm it works with 512x512 with my code. Tested on:

  1. Windows Nvidia
  2. Mac M series
  3. Intel Mac
anvoynov commented 8 months ago

Hi! You can transfer style from a real image with the newly added notebook: https://github.com/google/style-aligned/blob/main/style_aligned_transfer_sdxl.ipynb