Open BEpresent opened 1 year ago
update, also happening on a 3090 GPU
2023-04-13T20:05:33+0000 [ERROR] [runner:sd2:1] Application startup failed. Exiting.
/usr/local/lib/python3.10/dist-packages/transformers/models/clip/feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
warnings.warn(
2023-04-13T20:05:41+0000 [ERROR] [runner:sd2:1] An exception occurred while instantiating runner 'sd2', see details below:
2023-04-13T20:05:41+0000 [ERROR] [runner:sd2:1] Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/bentoml/_internal/runner/runner.py", line 293, in init_local
self._set_handle(LocalRunnerRef)
File "/usr/local/lib/python3.10/dist-packages/bentoml/_internal/runner/runner.py", line 139, in _set_handle
runner_handle = handle_class(self, *args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/bentoml/_internal/runner/runner_handle/local.py", line 24, in __init__
self._runnable = runner.runnable_class(**runner.runnable_init_params) # type: ignore
File "/usr/local/lib/python3.10/dist-packages/bentoml/_internal/frameworks/diffusers.py", line 443, in __init__
self.pipeline: diffusers.DiffusionPipeline = load_model(
File "/usr/local/lib/python3.10/dist-packages/bentoml/_internal/frameworks/diffusers.py", line 182, in load_model
pipeline = pipeline.to(device_id)
File "/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/pipeline_utils.py", line 626, in to
raise ValueError(
ValueError: It seems like you have activated sequential model offloading by calling `enable_sequential_cpu_offload`, but are now attempting to move the pipeline to GPU. This is not compatible with offloading. Please, move your pipeline `.to('cpu')` or consider removing the move altogether if you use sequential offloading.
Hi @BEpresent
I think there's a diffusers update breaking bentoml.diffusers
. We are going to fix this one. You can lock diffusers==0.13.1
for a temporal fix
Hi, I was following this example https://modelserving.com/blog/creating-stable-diffusion-20-service-with-bentoml-and-diffusers
or this by git clone of this example repo https://github.com/bentoml/diffusers-examples/tree/main/sd2
which results in a simple
service.py
file like this:After
bentoml serve service:svc --production
I get the following error (happens also with another custom model that I tried). It seems to be related toenable_sequential_cpu_offload
by HuggingFace.As general info, it runs on a GCS VM instance with T4 GPU - could this be the issue?