search

Vchitect / Vlogger

[CVPR2024] Make Your Dream A Vlog
Apache License 2.0
415 stars 42 forks source link

Need advice to setup to get a local working model of Stablediffusion #14

Open cercatore opened 7 months ago

cercatore commented 7 months ago

Hello This looks like a promising app and i like very much your effort in this. I am trying to develop a full app included frontend for a project of mine using vLogger. That is quite challenging However i ran into some issues I am unable to get a local copy of stablediffusion model Tried several ways with python

https://chat.openai.com/share/94d052f4-5b87-4848-b347-1982307917e7

This code works for OpenClip and showMaker, fails for sd-v1-5 Do you know any workaround for this? Thanks

zhuangshaobin commented 7 months ago

Sorry, I can't open the link in your question. It prompts me to enter my account and password.

cercatore commented 7 months ago

hello thanks for the reply my code is this

from transformers import AutoModel, AutoTokenizer

model_id = "openai/stablediffusion-v1-4"  # ID of the Stable Diffusion model

# Download tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id)

# Save the tokenizer and model
tokenizer.save_pretrained("/cache/tokenizer")
model.save_pretrained("/cache/model")
zhuangshaobin commented 7 months ago

If you want to call ip-adapter directly on the stable diffusion model, then you need to query the interface of ip-adapter in diffusers: https://huggingface.co/docs/diffusers/api/loaders/ip_adapter

The reason for the incompatibility may be that in order to add STEB, I rewrote the model code in the models folder, which may cause the code to be inconsistent with the interface in diffusers.