Open digiphd opened 1 year ago
Hey, @digiphd! Thanks for getting this on my radar. I'll have a chance to take a look during this coming week.
As a preliminary comment, I like the idea of being able to switch the VAE at runtime, although there will be a lot of work involved to adapt how we currently cache models.
P.S. If you're impatient, in the meantime, I think you could probably:
vae
directory with the contents from https://huggingface.co/stabilityai/sd-vae-ft-mse/tree/mainAlternatively, with your current setup, it's possible that if you set MODEL_PRECISION=""
and MODEL_REVISION=""
, you might get past that error by using full precision (but inference will be slower; nevertheless, maybe something useful in the interim).
Anyways, have a great weekend and we'll be in touch next week :grinning:
Hey @gadicc great, thanks for your suggestions I will give them ago! You're a legend!
Another thing I was wondering, was if docker-diffusers-api text-to-image supports negative keywords?
I did put it as an argument and it seemed to negatively affect the output images.
Yup! negative_prompt
modelInput, as it seems you worked out.
The modelInput
's are passed directly to the relevant diffusers' pipeline, so you can use whatever arguments are supported by that pipeline. I made this a little clearer in the README a few days ago with links to the common diffusers pipelines, as I admit it wasn't so obvious until then :sweat_smile:
There's also a note there now about using the lpw_stable_diffusion
pipeline which supports longer prompts and prompt weights.
Thanks for all the kind words! :raised_hands:
Hey @digiphd, I had a quick moment to try dreamlike-art/dreamlike-photoreal-2.0
and it works out the box for me, in both full and half precision. What version of docker-diffusers-api
are you using?
These worked for me:
$ python test.py txt2img --call-arg MODEL_ID="dreamlike-art/dreamlike-photoreal-2.0" --call-arg MODEL_PRECISION=""
$ python test.py txt2img --call-arg MODEL_ID="dreamlike-art/dreamlike-photoreal-2.0" --call-arg MODEL_PRECISION="fp16"
I just tried in the default "runtime" config. If you have this issue specifically in the -build-download
variant, let me know.
Hey hey!
So I am using some models that either have VAE baked in or require a separate VAE to be defined during inference like this:
when I either manually added the vae or used a model with a vae baked in for the
MODEL_ID
, I received the following error, for example with the modeldreamlike-art/dreamlike-photoreal-2.0
Line 382 in the inference function which looks like this:
images = pipeline(**model_inputs).images
Perhaps we need to add a .half() to the input somewhere, not sure where. though.
Any help would be greatly appreciated!
It's the last hurdle I am facing to be generating images.
IDEA: It would be awesome if we could define an optional VAE when making API call like this: