search

microsoft / DeepSpeed-MII

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
Apache License 2.0
1.85k stars 173 forks source link

stable diffusion fails looking for `SAFE_WEIGHTS_NAME` #159

Open lizelive opened 1 year ago

lizelive commented 1 year ago

ran into a few issues trying to run https://github.com/microsoft/DeepSpeed-MII/tree/main/examples/benchmark/txt2img

  1. need to set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
  2. ImportError: cannot import name 'SAFE_WEIGHTS_NAME' from 'transformers.utils' (/usr/local/lib/python3.8/dist-packages/transformers/utils/init.py)

i don't see an easy way to work around SAFE_WEIGHTS_NAME

to reproduce docker run -it --rm --gpus=all --ipc=host -p 127.0.0.1:8888:8888 -v root-cache:/root/.cache --ulimit=memlock=-1 --ulimit=stack=67108864 nvcr.io/nvidia/pytorch:23.02-py3 jupyter lab --allow-root --ip=* --port=8888 --no-browser --NotebookApp.token='' --NotebookApp.allow_origin='*' and then run pip install pytorch-lightning diffusers transformers scipy safetensors accelerate deepspeed[sd] deepspeed-mii

here is the error

root@58da005d6d1f:/workspace# PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python python ./cool.py 
Traceback (most recent call last):
  File "./cool.py", line 7, in <module>
    import diffusers
  File "/usr/local/lib/python3.8/dist-packages/diffusers/__init__.py", line 55, in <module>
    from .pipelines import (
  File "/usr/local/lib/python3.8/dist-packages/diffusers/pipelines/__init__.py", line 18, in <module>
    from .dance_diffusion import DanceDiffusionPipeline
  File "/usr/local/lib/python3.8/dist-packages/diffusers/pipelines/dance_diffusion/__init__.py", line 1, in <module>
    from .pipeline_dance_diffusion import DanceDiffusionPipeline
  File "/usr/local/lib/python3.8/dist-packages/diffusers/pipelines/dance_diffusion/pipeline_dance_diffusion.py", line 21, in <module>
    from ..pipeline_utils import AudioPipelineOutput, DiffusionPipeline
  File "/usr/local/lib/python3.8/dist-packages/diffusers/pipelines/pipeline_utils.py", line 64, in <module>
    from transformers.utils import SAFE_WEIGHTS_NAME as TRANSFORMERS_SAFE_WEIGHTS_NAME
ImportError: cannot import name 'SAFE_WEIGHTS_NAME' from 'transformers.utils' (/usr/local/lib/python3.8/dist-packages/transformers/utils/__init__.py)

showing version numbers

Name: pytorch-lightning
Version: 1.9.4
Summary: PyTorch Lightning is the lightweight PyTorch wrapper for ML researchers. Scale your models. Write less boilerplate.
Home-page: https://github.com/Lightning-AI/lightning
Author: Lightning AI et al.
Author-email: pytorch@lightning.ai
License: Apache-2.0
Location: /usr/local/lib/python3.8/dist-packages
Requires: lightning-utilities, tqdm, torch, PyYAML, typing-extensions, packaging, torchmetrics, fsspec, numpy
Required-by: 
---
Name: diffusers
Version: 0.14.0
Summary: Diffusers
Home-page: https://github.com/huggingface/diffusers
Author: The HuggingFace team
Author-email: patrick@huggingface.co
License: Apache
Location: /usr/local/lib/python3.8/dist-packages
Requires: importlib-metadata, numpy, filelock, huggingface-hub, regex, Pillow, requests
Required-by: 
---
Name: transformers
Version: 4.21.2
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache
Location: /usr/local/lib/python3.8/dist-packages
Requires: tqdm, pyyaml, filelock, tokenizers, huggingface-hub, numpy, packaging, regex, requests
Required-by: deepspeed-mii
---
Name: scipy
Version: 1.6.3
Summary: SciPy: Scientific Library for Python
Home-page: https://www.scipy.org/
Author: 
Author-email: 
License: BSD
Location: /usr/local/lib/python3.8/dist-packages
Requires: numpy
Required-by: xgboost, treelite, treelite-runtime, scikit-learn, pytorch-quantization, librosa, cuml
---
Name: safetensors
Version: 0.3.0
Summary: Fast and Safe Tensor serialization
Home-page: https://github.com/huggingface/safetensors
Author: 
Author-email: 
License: Apache License 2.0
Location: /usr/local/lib/python3.8/dist-packages
Requires: 
Required-by: 
---
Name: accelerate
Version: 0.17.1
Summary: Accelerate
Home-page: https://github.com/huggingface/accelerate
Author: The HuggingFace team
Author-email: sylvain@huggingface.co
License: Apache
Location: /usr/local/lib/python3.8/dist-packages
Requires: psutil, torch, numpy, packaging, pyyaml
Required-by: 
---
Name: deepspeed
Version: 0.7.3
Summary: DeepSpeed library
Home-page: http://deepspeed.ai/
Author: DeepSpeed Team
Author-email: deepspeed@microsoft.com
License: MIT
Location: /usr/local/lib/python3.8/dist-packages
Requires: py-cpuinfo, ninja, psutil, packaging, numpy, tqdm, pydantic, hjson, torch
Required-by: deepspeed-mii
---
Name: deepspeed-mii
Version: 0.0.2
Summary: deepspeed mii
Home-page: http://deepspeed.ai/
Author: DeepSpeed Team
Author-email: deepspeed-mii@microsoft.com
License: 
Location: /usr/local/lib/python3.8/dist-packages
Requires: transformers, grpcio-tools, grpcio, asyncio, pydantic, torch, deepspeed
Required-by: 
Note: you may need to restart the kernel to use updated packages.
tianleiwu commented 1 year ago

@lizelive, The error means you need upgrade your transformers package. The linked page has description about package versions:

deepspeed==0.7.5
deepspeed-mii==0.0.3
diffusers==0.7.1
transformers==4.24.0
triton==2.0.0.dev20221005  (This does not exist, and you can use 2.0.0.dev20221031 instead)
Ubuntu 20.04.4 LTS
Python 3.9.15

You can pip install one by one like pip install deepspeed==0.7.5.

I can run the benchmark script mii-sd.py with above packages and get similar latency (with torch 1.13.1 and CUDA 11.7, although the page recommend torch 1.12.1 + cuda 11.6).