search

AUTOMATIC1111 / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
141.39k stars 26.72k forks source link

RuntimeError: Placeholder storage has not been allocated on MPS device! #1708

Closed Poloman12 closed 2 years ago

Poloman12 commented 2 years ago

Hello! I receive an error trying to use the "Restore Faces" feature and enable "CodeFormer" or "GFPGAN". This worked before but after resetting my Mac and setup stable diffusion again I get this error.

Full error:

Traceback (most recent call last): File "/Users/fk/Documents/stable-diffusion-webui/modules/ui.py", line 184, in f res = list(func(*args, kwargs)) File "/Users/fk/Documents/stable-diffusion-webui/webui.py", line 64, in f res = func(*args, *kwargs) File "/Users/fk/Documents/stable-diffusion-webui/modules/txt2img.py", line 43, in txt2img processed = process_images(p) File "/Users/fk/Documents/stable-diffusion-webui/modules/processing.py", line 400, in process_images x_sample = modules.face_restoration.restore_faces(x_sample) File "/Users/fk/Documents/stable-diffusion-webui/modules/face_restoration.py", line 19, in restore_faces return face_restorer.restore(np_image) File "/Users/fk/Documents/stable-diffusion-webui/modules/codeformer_model.py", line 93, in restore self.face_helper.get_face_landmarks_5(only_center_face=False, resize=640, eye_dist_threshold=5) File "/Users/fk/Documents/stable-diffusion-webui/repositories/CodeFormer/facelib/utils/face_restoration_helper.py", line 154, in get_face_landmarks_5 bboxes = self.face_det.detect_faces(input_img) File "/Users/fk/Documents/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/retinaface/retinaface.py", line 209, in detect_faces loc, conf, landmarks, priors = self.__detect_faces(image) File "/Users/fk/Documents/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/retinaface/retinaface.py", line 156, in __detect_faces loc, conf, landmarks = self(inputs) File "/Users/fk/miniconda/envs/web-ui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(input, kwargs) File "/Users/fk/Documents/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/retinaface/retinaface.py", line 121, in forward out = self.body(inputs) File "/Users/fk/miniconda/envs/web-ui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(*input, *kwargs) File "/Users/fk/miniconda/envs/web-ui/lib/python3.10/site-packages/torchvision/models/_utils.py", line 69, in forward x = module(x) File "/Users/fk/miniconda/envs/web-ui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(input, **kwargs) File "/Users/fk/miniconda/envs/web-ui/lib/python3.10/site-packages/torch/nn/modules/batchnorm.py", line 171, in forward return F.batch_norm( File "/Users/fk/miniconda/envs/web-ui/lib/python3.10/site-packages/torch/nn/functional.py", line 2446, in batch_norm return torch.batch_norm( RuntimeError: Placeholder storage has not been allocated on MPS device!

Thanks for the help!

Gitterman69 commented 2 years ago

Same error while inpainting on M1 MBP on Monterey


=============================================
=============================================
=============================================
=============================================
Warning: caught exception 'Torch not compiled with CUDA enabled', memory monitor disabled
Moving sd-v1-4.ckpt from /Users/bamboozle/stable-diffusion-webui/models to /Users/bamboozle/stable-diffusion-webui/models/Stable-diffusion.
WARNING:root:Pytorch pre-release version 1.13.0.dev20220922 - assuming intent to test it
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Loading weights [7460a6fa] from /Users/bamboozle/stable-diffusion-webui/models/Stable-diffusion/sd-v1-4.ckpt
Global Step: 470000
Model loaded.
Loaded a total of 0 textual inversion embeddings.
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
100%|███████████████████████████████████████████| 20/20 [00:34<00:00,  1.71s/it]
Total progress: 100%|███████████████████████████| 20/20 [00:31<00:00,  1.58s/it]
100%|███████████████████████████████████████████| 20/20 [06:24<00:00, 19.21s/it]
Total progress: 100%|███████████████████████████| 20/20 [06:08<00:00, 18.43s/it]
100%|█████████████████████████████████████████████| 8/8 [02:22<00:00, 17.78s/it]
Total progress: 100%|█████████████████████████████| 8/8 [02:04<00:00, 15.54s/it]100%|███████████████████████████████████████████| 16/16 [00:30<00:00,  1.92s/it]
Total progress: 100%|███████████████████████████| 16/16 [00:26<00:00,  1.63s/it]
  0%|                                                    | 0/16 [00:00<?, ?it/s]100%|███████████████████████████████████████| 16/16 [4:29:45<00:00, 1011.58s/it]
Total progress: 100%|███████████████████████| 16/16 [4:45:57<00:00, 1072.31s/it]
100%|█████████████████████████████████████████████| 8/8 [01:53<00:00, 14.22s/it]
Total progress: 100%|█████████████████████████████| 8/8 [01:33<00:00, 11.70s/it]
100%|█████████████████████████████████████████████| 8/8 [02:28<00:00, 18.52s/it]
Total progress: 100%|█████████████████████████████| 8/8 [02:07<00:00, 15.92s/it]
100%|███████████████████████████████████████████| 20/20 [00:44<00:00,  2.25s/it]
Total progress: 100%|███████████████████████████| 20/20 [00:31<00:00,  1.55s/it]
100%|███████████████████████████████████████████| 14/14 [03:58<00:00, 17.03s/it]
Downloading: "https://github.com/sczhou/CodeFormer/releases/download/v0.1.0/codeformer.pth" to /Users/bamboozle/stable-diffusion-webui/models/Codeformer/codeformer-v0.1.0.pth

100%|████████████████████████████████████████| 359M/359M [00:10<00:00, 36.8MB/s]
Downloading: "https://github.com/sczhou/CodeFormer/releases/download/v0.1.0/detection_Resnet50_Final.pth" to /Users/bamboozle/stable-diffusion-webui/repositories/CodeFormer/weights/facelib/detection_Resnet50_Final.pth

100%|████████████████████████████████████████| 104M/104M [00:05<00:00, 20.3MB/s]
Downloading: "https://github.com/sczhou/CodeFormer/releases/download/v0.1.0/parsing_parsenet.pth" to /Users/bamboozle/stable-diffusion-webui/repositories/CodeFormer/weights/facelib/parsing_parsenet.pth

100%|██████████████████████████████████████| 81.4M/81.4M [00:03<00:00, 24.4MB/s]
Error completing request
Arguments: (1, 'man dancing with the death by magritte, surreal, detailed, scarlet red, oil painting, high resolution, 4k', '', 'None', 'None', None, {'image': <PIL.Image.Image image mode=RGBA size=640x832 at 0x3694E5B40>, 'mask': <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=640x832 at 0x3694E4160>}, None, None, 0, 18, 0, 4, 1, True, False, 1, 1, 10.5, 0.75, -1.0, -1.0, 0, 0, 0, False, 832, 640, 0, False, 32, 0, '', '', 0, '', '', 1, 50, 0, False, 4, 1, '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, None, '', '<p style="margin-bottom:0.75em">Will upscale the image to twice the dimensions; use width and height sliders to set tile size</p>', 64, 0, 1, '', 4, '', True, False) {}
Traceback (most recent call last):
  File "/Users/bamboozle/stable-diffusion-webui/modules/ui.py", line 184, in f
    res = list(func(*args, **kwargs))
  File "/Users/bamboozle/stable-diffusion-webui/webui.py", line 64, in f
    res = func(*args, **kwargs)
  File "/Users/bamboozle/stable-diffusion-webui/modules/img2img.py", line 124, in img2img
    processed = process_images(p)
  File "/Users/bamboozle/stable-diffusion-webui/modules/processing.py", line 400, in process_images
    x_sample = modules.face_restoration.restore_faces(x_sample)
  File "/Users/bamboozle/stable-diffusion-webui/modules/face_restoration.py", line 19, in restore_faces
    return face_restorer.restore(np_image)
  File "/Users/bamboozle/stable-diffusion-webui/modules/codeformer_model.py", line 93, in restore
    self.face_helper.get_face_landmarks_5(only_center_face=False, resize=640, eye_dist_threshold=5)
  File "/Users/bamboozle/stable-diffusion-webui/repositories/CodeFormer/facelib/utils/face_restoration_helper.py", line 154, in get_face_landmarks_5
    bboxes = self.face_det.detect_faces(input_img)
  File "/Users/bamboozle/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/retinaface/retinaface.py", line 209, in detect_faces
    loc, conf, landmarks, priors = self.__detect_faces(image)
  File "/Users/bamboozle/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/retinaface/retinaface.py", line 156, in __detect_faces
    loc, conf, landmarks = self(inputs)
  File "/Users/bamboozle/opt/miniconda3/envs/web-ui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/bamboozle/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/retinaface/retinaface.py", line 121, in forward
    out = self.body(inputs)
  File "/Users/bamboozle/opt/miniconda3/envs/web-ui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/bamboozle/opt/miniconda3/envs/web-ui/lib/python3.10/site-packages/torchvision/models/_utils.py", line 69, in forward
    x = module(x)
  File "/Users/bamboozle/opt/miniconda3/envs/web-ui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/bamboozle/opt/miniconda3/envs/web-ui/lib/python3.10/site-packages/torch/nn/modules/batchnorm.py", line 171, in forward
    return F.batch_norm(
  File "/Users/bamboozle/opt/miniconda3/envs/web-ui/lib/python3.10/site-packages/torch/nn/functional.py", line 2446, in batch_norm
    return torch.batch_norm(
RuntimeError: Placeholder storage has not been allocated on MPS device!
brkirch commented 2 years ago

If you are starting the web UI using launch.py or webui.py then add --use-cpu GFPGAN CodeFormer to the end of your command line arguments. If you are using run_webui_mac.sh then open it in a text editor and change

python webui.py --precision full --no-half --opt-split-attention-v1

to

python webui.py --precision full --no-half --opt-split-attention-v1 --use-cpu GFPGAN CodeFormer

There could also be some other modules that need to be listed to work (ESRGAN needed to use CPU when I last tried it, for example) but that depends on the versions of the modules you have installed (and especially which nightly build of PyTorch is installed). If you find that any other module isn’t working correctly, you can try adding it to the --use-cpu option and see if it helps.

NeoTrace82 commented 2 years ago

same problem, after changing the settings in run_webui_mac.sh it does something but then fails with the following error

Arguments: ('test robot', '', 'None', 'None', 20, 0, True, False, 1, 1, 7, -1.0, -1.0, 0, 0, 0, False, 512, 512, False, False, 0.7, 0, False, False, None, '', 1, '', 4, '', True, False) {} Traceback (most recent call last): File "/Users/thomas/SD_Test/stable-diffusion-webui/modules/ui.py", line 184, in f res = list(func(*args, **kwargs)) File "/Users/thomas/SD_Test/stable-diffusion-webui/webui.py", line 65, in f res = func(*args, **kwargs) File "/Users/thomas/SD_Test/stable-diffusion-webui/modules/txt2img.py", line 43, in txt2img processed = process_images(p) File "/Users/thomas/SD_Test/stable-diffusion-webui/modules/processing.py", line 399, in process_images x_sample = modules.face_restoration.restore_faces(x_sample) File "/Users/thomas/SD_Test/stable-diffusion-webui/modules/face_restoration.py", line 19, in restore_faces return face_restorer.restore(np_image) File "/Users/thomas/SD_Test/stable-diffusion-webui/modules/codeformer_model.py", line 93, in restore self.face_helper.get_face_landmarks_5(only_center_face=False, resize=640, eye_dist_threshold=5) File "/Users/thomas/SD_Test/stable-diffusion-webui/repositories/CodeFormer/facelib/utils/face_restoration_helper.py", line 154, in get_face_landmarks_5 bboxes = self.face_det.detect_faces(input_img) File "/Users/thomas/SD_Test/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/retinaface/retinaface.py", line 231, in detect_faces keep = py_cpu_nms(bounding_boxes, nms_threshold) File "/Users/thomas/SD_Test/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/retinaface/retinaface_utils.py", line 41, in py_cpu_nms keep = torchvision.ops.nms( File "/Users/thomas/miniconda/envs/web-ui/lib/python3.10/site-packages/torchvision/ops/boxes.py", line 40, in nms _assert_has_ops() File "/Users/thomas/miniconda/envs/web-ui/lib/python3.10/site-packages/torchvision/extension.py", line 32, in _assert_has_ops raise RuntimeError( RuntimeError: Couldn't load custom C++ ops. This can happen if your PyTorch and torchvision versions are incompatible, or if you had errors while compiling torchvision from source. For further information on the compatible versions, check https://github.com/pytorch/vision#installation for the compatibility matrix. Please check your PyTorch version with torch.__version__ and your torchvision version with torchvision.__version__ and verify if they are compatible, and if not please reinstall torchvision so that it matches your PyTorch install.

asuming the last part is the problem but I have no clue which versions are compatible and how to check or install them tried to import torch in python but that does come up with an error too

NeoTrace82 commented 2 years ago

ran the install script again to check and it says it installed torch-1.12.1 and torchvision-0.13.1 so that seems ok

Successfully installed MarkupSafe-2.1.1 Pillow-9.2.0 PyWavelets-1.4.1 absl-py-1.2.0 addict-2.4.0 aiohttp-3.8.3 aiosignal-1.2.0 analytics-python-1.4.0 antlr4-python3-runtime-4.9.3 anyio-3.6.1 async-timeout-4.0.2 attrs-22.1.0 backoff-1.10.0 basicsr-1.4.2 bcrypt-4.0.0 cachetools-5.2.0 cffi-1.15.1 chardet-4.0.0 charset-normalizer-2.1.1 clean-fid-0.1.31 click-8.1.3 coloredlogs-15.0.1 contourpy-1.0.5 cryptography-38.0.1 cycler-0.11.0 diffusers-0.3.0 einops-0.5.0 facexlib-0.2.5 fairscale-0.4.4 fastapi-0.85.0 ffmpy-0.3.0 filelock-3.8.0 filterpy-1.4.5 flatbuffers-22.9.24 font-roboto-0.0.1 fonts-0.0.3 fonttools-4.37.4 frozenlist-1.3.1 fsspec-2022.8.2 future-0.18.2 gfpgan-1.3.8 google-auth-2.12.0 google-auth-oauthlib-0.4.6 gradio-3.4b3 grpcio-1.49.1 h11-0.12.0 httpcore-0.15.0 httpx-0.23.0 huggingface-hub-0.10.0 humanfriendly-10.0 idna-2.10 imageio-2.22.1 importlib-metadata-5.0.0 invisible-watermark-0.1.5 jinja2-3.1.2 jsonmerge-1.8.0 jsonschema-4.16.0 kiwisolver-1.4.4 kornia-0.6.7 lark-1.1.2 linkify-it-py-1.0.3 llvmlite-0.39.1 lmdb-1.3.0 markdown-3.4.1 markdown-it-py-2.1.0 matplotlib-3.6.0 mdit-py-plugins-0.3.1 mdurl-0.1.2 monotonic-1.6 mpmath-1.2.1 multidict-6.0.2 networkx-2.8.7 numba-0.56.2 numpy-1.23.3 oauthlib-3.2.1 omegaconf-2.2.3 onnx-1.12.0 onnxruntime-1.12.1 opencv-python-4.6.0.66 orjson-3.8.0 packaging-21.3 pandas-1.5.0 paramiko-2.11.0 piexif-1.1.3 protobuf-3.19.6 pyDeprecate-0.3.2 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pycryptodome-3.15.0 pydantic-1.10.2 pydub-0.25.1 pynacl-1.5.0 pyparsing-3.0.9 pyrsistent-0.18.1 python-dateutil-2.8.2 python-multipart-0.0.5 pytorch_lightning-1.7.7 pytz-2022.4 pyyaml-6.0 realesrgan-0.3.0 regex-2022.9.13 requests-2.25.1 requests-oauthlib-1.3.1 resize-right-0.0.2 rfc3986-1.5.0 rsa-4.9 scikit-image-0.19.3 scipy-1.9.1 setuptools-59.8.0 six-1.16.0 sniffio-1.3.0 starlette-0.20.4 sympy-1.11.1 tb-nightly-2.11.0a20221005 tensorboard-2.10.1 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tifffile-2022.8.12 timm-0.4.12 tokenizers-0.12.1 torch-1.12.1 torchdiffeq-0.2.3 torchmetrics-0.10.0 torchvision-0.13.1 tqdm-4.64.1 transformers-4.19.2 typing-extensions-4.3.0 uc-micro-py-1.0.1 urllib3-1.26.12 uvicorn-0.18.3

brkirch commented 2 years ago

That’s just what is installed by Conda, the torch install gets replaced later in the script by a different version. It looks like the install script uses pip to install the newest nightly of torchvision despite installing an older version of torch which causes a version mismatch error.

To fix it, open a new terminal window and run: conda activate web-ui pip install --pre torch==1.13.0.dev20220924 torchvision==0.14.0.dev20220924 -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html --no-deps

@dylancl You may want to update setup_mac.sh to fix the problems mentioned in this issue.

dylancl commented 2 years ago

@brkirch fixed, thanks.

NeoTrace82 commented 2 years ago

conda activate web-ui

how to run conda on M1? might be so simple but I dont know, did just run the script which uses the command but running it from the terminal gives me the output that it doesn't know the command

NeoTrace82 commented 2 years ago

never mind got it working, just looked at the script... im stupid :)

thomas@MacBook-Pro-von-Thomas ~ % /usr/bin/env bash -l 
The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
(base) MacBook-Pro-von-Thomas:~ thomas$ conda activate web-ui
WARNING: overwriting environment variables set in the machine
overwriting variable PYTORCH_ENABLE_MPS_FALLBACK
(web-ui) MacBook-Pro-von-Thomas:~ thomas$ pip install --pre torch==1.13.0.dev20220924 torchvision==0.14.0.dev20220924 -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html --no-deps