ValueError: Multiple file extensions found at ./models/coreml-stable-diffusion-v1-4_original_packages.Cannot infer resource type from contents.

LukeLIN-web commented 1 month ago

Describe the bug

First problem is that, I run https://huggingface.co/docs/diffusers/v0.30.3/en/optimization/coreml#stable-diffusion-core-ml-checkpoints failed.

Second problem is that, For example, if you want to use [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5): this link is 404 not found.

Reproduction

from huggingface_hub import snapshot_download
from pathlib import Path

repo_id = "apple/coreml-stable-diffusion-v1-4"
variant = "original/packages"

model_path = Path("./models") / (repo_id.split("/")[-1] + "_" + variant.replace("/", "_"))
snapshot_download(repo_id, allow_patterns=f"{variant}/*", local_dir=model_path, local_dir_use_symlinks=False)
print(f"Model downloaded at {model_path}")

python -m python_coreml_stable_diffusion.pipeline --prompt "a photo of an astronaut riding a horse on mars" -i models/coreml-stable-diffusion-v1-4_original_packages -o </path/to/output/image> --compute-unit CPU_AND_GPU --seed 93

Logs

~/Desktop/stable-diffusion-videos % python -m python_coreml_stable_diffusion.pipeline --prompt "a photo of an astronaut riding a horse on mars" -i ./models/coreml-stable-diffusion-v1-4_original_packages -o ./output --compute-unit CPU_AND_GPU --seed 93
WARNING:coremltools:Torch version 2.4.1 has not been tested with coremltools. You may run into unexpected errors. Torch 2.4.0 is the most recent version that has been tested.
INFO:__main__:Setting random seed to 93
INFO:__main__:Initializing PyTorch pipe for reference configuration
Keyword arguments {'use_auth_token': True} are not expected by StableDiffusionPipeline and will be ignored.
Loading pipeline components...:  86%|███████████████████████████████████████████████████████████████████▋           | 6/7 [00:00<00:00, 25.55it/s]/Users/linj/miniconda3/envs/sdv/lib/python3.9/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  warnings.warn(
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 25.59it/s]
INFO:__main__:Removed PyTorch pipe to reduce peak memory consumption
INFO:__main__:Loading Core ML models in memory from ./models/coreml-stable-diffusion-v1-4_original_packages
Traceback (most recent call last):
  File "/Users/linj/miniconda3/envs/sdv/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/linj/miniconda3/envs/sdv/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/linj/miniconda3/envs/sdv/lib/python3.9/site-packages/python_coreml_stable_diffusion/pipeline.py", line 825, in <module>
    main(args)
  File "/Users/linj/miniconda3/envs/sdv/lib/python3.9/site-packages/python_coreml_stable_diffusion/pipeline.py", line 720, in main
    coreml_pipe = get_coreml_pipe(
  File "/Users/linj/miniconda3/envs/sdv/lib/python3.9/site-packages/python_coreml_stable_diffusion/pipeline.py", line 653, in get_coreml_pipe
    coreml_pipe_kwargs.update({
  File "/Users/linj/miniconda3/envs/sdv/lib/python3.9/site-packages/python_coreml_stable_diffusion/pipeline.py", line 654, in <dictcomp>
    model_name: _load_mlpackage(
  File "/Users/linj/miniconda3/envs/sdv/lib/python3.9/site-packages/python_coreml_stable_diffusion/coreml_model.py", line 168, in _load_mlpackage
    sources = get_resource_type(mlpackages_dir)
  File "/Users/linj/miniconda3/envs/sdv/lib/python3.9/site-packages/python_coreml_stable_diffusion/coreml_model.py", line 142, in get_resource_type
    raise ValueError(f'Multiple file extensions found at {resources_dir}.'
ValueError: Multiple file extensions found at ./models/coreml-stable-diffusion-v1-4_original_packages.Cannot infer resource type from contents.

System Info

🤗 Diffusers version: 0.30.2
Platform: macOS-14.6.1-arm64-arm-64bit
Running on Google Colab?: No
Python version: 3.12.5
PyTorch version (GPU?): 2.4.1 (False)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Huggingface_hub version: 0.24.7
Transformers version: 4.44.2
Accelerate version: 0.34.2
PEFT version: not installed
Bitsandbytes version: not installed
Safetensors version: 0.4.5
xFormers version: not installed
Accelerator: Apple M2
Using GPU in script?: no
Using distributed or parallel set-up in script?: no

Who can help?

@DN6 @a-r-r-o-w

LukeLIN-web commented 1 month ago

Find a solution in https://github.com/apple/ml-stable-diffusion/issues/163#issuecomment-2361337478
We need a more specific file path. ./models/coreml-stable-diffusion-v1-4_original_packages/original/packages

LukeLIN-web commented 1 month ago

And https://github.com/huggingface/diffusers/blob/2b443a5d621bd65f5cbf854195aef29cedd24058/docs/source/en/api/pipelines/text_to_video_zero.md#text-to-video It should has import imageio

asomoza commented 1 month ago

Hi, I can't help with the apple models but you seem to have resolved it.

About the the other two issues:

Second problem is that, For example, if you want to use [runwayml/stable-diffusion-v1-5]

Runwayml pulled off all their models from HF, so none of the links that have runwayml will work, this was a big news not long ago.

It should has import imageio

You're right.

Both of this means that we need to update the docs, so thanks a lot for reporting them.

cc: @stevhliu

stevhliu commented 1 month ago

Thanks for reporting!

Would you like to open a PR with the correct path and also import imageio?
The runwayml/stable-diffusion-v1-5 checkpoint has been updated to stable-diffusion-v1-5/stable-diffusion-v1-5 on the main version of the docs.

huggingface / diffusers