search

huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
https://huggingface.co/docs/diffusers
Apache License 2.0
25.18k stars 5.21k forks source link

[Community] DreamArtist : One-Shot Text-to-Image Generation For Diffusers #1290

Open adhikjoshi opened 1 year ago

adhikjoshi commented 1 year ago

Is your feature request related to a problem? Please describe. There is new, One-Shot Text-to-Image for Texual Inversion

Twitter Link

There is implementation for Automatic1111, can we have same for diffusers?

Github Code

Describe the solution you'd like

DreamArtist for Dreambooth Diffusers

aandyw commented 1 year ago

Very interested in this issue. I would like to take this on with some guidance if possible :) .

isamu-isozaki commented 1 year ago

@Pie31415 Nice! I might implement it in my fork so you can check that if you want. The main changes for this work seems to be

  1. Using negative prompts too to learn what the image is not at the same time as the regular tokens
  2. Using a discriminator instead of the noise loss
aandyw commented 1 year ago

@isamu-isozaki Hey, sounds good. Thanks for the first 2 points. Also, do you mind if we work on the issue together? I've just started learning about diffusers so my knowledge is a little limited. Would you possibly be able to give me some guidance or resources to get started?

aandyw commented 1 year ago

Also, this maybe a dumb question, but I'm assuming we're implementing a new pipeline with the 2 previously mentioned changes instead of a new model?

isamu-isozaki commented 1 year ago

@Pie31415 For sure! I guess for resources maybe the textual inversion paper but I was thinking that we can copy the code from the textual inversion, and just make slight changes to do this job. We might do our custom pipeline but pipelines are mainly for inference and not training. So I'm thinking first we make the training script-> pipeline.

isamu-isozaki commented 1 year ago

I can try setting up some code by the end of the day

aandyw commented 1 year ago

@isamu-isozaki Thanks a lot for the help. Would you like to move this conversation to discord? I think it'll be more convenient for communication.

isamu-isozaki commented 1 year ago

@Pie31415 yup sounds good. My name's Chad Kensington on discord. I'm in the Huggingface discord server.