search

LaurentMazare / diffusers-rs

An implementation of the diffusers api in Rust
Apache License 2.0
539 stars 55 forks source link

Implement Img2Img #3

Closed siriux closed 2 years ago

siriux commented 2 years ago

I'm going to start understanding the Img2Img pipeline and what is needed to port it.

Also, are you interested in keep growing the SD example or you prefer to keep it simple. I'm asking because for my use case of building an SD based image editor I'm going to need to create a separate library tailed for my needs, but maybe is interesting too also include some features like Img2Img in the SD example.

LaurentMazare commented 2 years ago

I was certainly planning on supporting img2img so glad if it can be added. I would think that it can be made into a separate example to start with, it would duplicate a bit of code but probably not that much, I've started moving some of the stable diffusion specific bits in src/pipelines/stable_diffusion.rs so that they can be shared between the two sides.

siriux commented 2 years ago

I got img2img working on my SD library https://github.com/siriux/stable-diffusion-rs/blob/main/examples/main.rs

It's much more complex than I initially though, and I had to debug many things until I got it working, but now it seems to generate identical images to Automatic1111 with the same parameters in all the examples that I tried.

To try it, the changes from this PR are needed https://github.com/LaurentMazare/diffusers-rs/pull/6

I want to work a little bit more to get inpainting working using the base 1.5 model as well as supporting the fine-tuned inpainting model.

And then I think we can port the changes to your example (before my library becomes an actual library and starts diverging too much) to cover the main basis of what SD is basically capable of.

LaurentMazare commented 2 years ago

I've added some examples using an img2img pipeline (readme) as well as some inpainting pipeline (readme). Both seem to work well on the examples that I've tried so I'm closing this issue for now, feel free to re-open if you find any issue, or have other suggestions of SD based pipelines that could be added.