search

YingqingHe / ScaleCrafter

[ICLR 2024 Spotlight] Official implementation of ScaleCrafter for higher-resolution visual generation at inference time.
487 stars 29 forks source link

Cannot use SD 1.5 and SD 2.1 models #9

Open SoftologyPro opened 11 months ago

SoftologyPro commented 11 months ago

If I try and use SD 1.5 with python text2image_xl.py --pretrained_model_name_or_path runwayml/stable-diffusion-v1-5 --validation_prompt "roses in the rain" --seed 1500289018 --config .\configs\sd1.5_2048x2048.yaml --logging_dir logs I get the error OSError: Can't load tokenizer for 'runwayml/stable-diffusion-v1-5'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'runwayml/stable-diffusion-v1-5' is the correct path to a directory containing all relevant files for a CLIPTokenizer tokenizer.

If I try and use SD 2.1 with python text2image_xl.py --pretrained_model_name_or_path stabilityai/stable-diffusion-2-1-base --validation_prompt "roses in the rain" --seed 470116415 --config .\configs\sd2.1_2048x2048.yaml --logging_dir logs I get the error OSError: Can't load tokenizer for 'stabilityai/stable-diffusion-2-1-base'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'stabilityai/stable-diffusion-2-1-base' is the correct path to a directory containing all relevant files for a CLIPTokenizer tokenizer.

If I try and use SD XL with python text2image_xl.py --pretrained_model_name_or_path stabilityai/stable-diffusion-xl-base-1.0 --validation_prompt "roses in the rain" --seed 694038880 --config .\configs\sdxl_2048x2048.yaml --logging_dir logs it works fine.

roses in the rain_sdxl_694038880

What am I missing to use the 1.5 and 2.1 models?

ssyang1999 commented 11 months ago

Hi. Could you please try to download the SD 1.5 and SD 2.1 checkpoints to your local device and use a local directory? Maybe the huggingface repo name given here is no longer valid.

For example, download SD 1.5 weights: git-lfs clone https://huggingface.co/runwayml/stable-diffusion-v1-5

SoftologyPro commented 11 months ago

OK, so I do a full clone of the 1.5 and 2.1 huggingface model repositories. What local directory name should I use?

ssyang1999 commented 11 months ago

For example, now the directory of cloned SD 1.5 is /data/weights/stable-diffusion-v1-5. Then you can use the absolute directory /data/weights/stable-diffusion-v1-5 to set the parameter --pretrained_model_name_or_path.

SoftologyPro commented 11 months ago

I created ran these 2 commands under the models subdirectory

git clone https://huggingface.co/runwayml/stable-diffusion-v1-5
git clone https://huggingface.co/stabilityai/stable-diffusion-2-1-base

and changed the parameter to --pretrained_model_name_or_path models/stable-diffusion-v1-5 but I still get OSError: Can't load tokenizer for 'models/stable-diffusion-v1-5'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'models/stable-diffusion-v1-5' is the correct path to a directory containing all relevant files for a CLIPTokenizer tokenizer. That is the right path now. Also the clones take up 158 GB worth of files. Do we really need all of that? Or are there only a few files from both required?