Apple M1 / M2 / MPS support

[ormedo]

Hi!

I just downladed de proyect and try to build and deploy the docker on my M1. I always get the same error. [+] Building 163.5s (15/44)
=> [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 6.90kB 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/pytorch/pytorch:1.12.1-cuda11.3-cudnn8-runtime 1.5s [+] Building 163.6s (15/44)
=> => transferring context: 64.22kB 0.0s => CACHED [base 1/5] FROM docker.io/pytorch/pytorch:1.12.1-cuda11.3-cudnn8-runtime@sha256:0bc0971dc8ae319af610d493aced87df46255c9508a8b9e9bc365f11a56e7b75 0.0s => [base 2/5] RUN if [ -n "" ] ; then echo quit | openssl s_client -proxy $(echo | cut -b 8-) -servername google.com -connect google.com:443 -showcerts | sed 'H;1h; 0.3s => [base 3/5] RUN apt-get update 14.3s => [base 4/5] RUN apt-get install -yqq git 27.6s [+] Building 1320.8s (18/44)
=> [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 6.90kB 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/pytorch/pytorch:1.12.1-cuda11.3-cudnn8-runtime 1.5s => [internal] load build context 0.0s => => transferring context: 64.22kB 0.0s => CACHED [base 1/5] FROM docker.io/pytorch/pytorch:1.12.1-cuda11.3-cudnn8-runtime@sha256:0bc0971dc8ae319af610d493aced87df46255c9508a8b9e9bc365f11a56e7b75 0.0s => [base 2/5] RUN if [ -n "" ] ; then echo quit | openssl s_client -proxy $(echo | cut -b 8-) -servername google.com -connect google.com:443 -showcerts | sed 'H;1h; 0.3s => [base 3/5] RUN apt-get update 14.3s => [base 4/5] RUN apt-get install -yqq git 27.6s => [base 5/5] RUN apt-get install -yqq zstd 8.3s => [output 1/32] RUN mkdir /api 0.5s => [patchmatch 1/3] WORKDIR /tmp 0.0s => [patchmatch 2/3] COPY scripts/patchmatch-setup.sh . 0.0s => [patchmatch 3/3] RUN sh patchmatch-setup.sh 0.4s => [output 2/32] WORKDIR /api 0.0s => [output 3/32] RUN conda update -n base -c defaults conda 101.1s => [output 4/32] RUN conda create -n xformers python=3.10 33.9s => [output 5/32] RUN python --version 6.3s => ERROR [output 6/32] RUN conda install -c pytorch -c conda-forge cudatoolkit=11.6 pytorch=1.12.1 1126.9s

[output 6/32] RUN conda install -c pytorch -c conda-forge cudatoolkit=11.6 pytorch=1.12.1:

14 9.049 Collecting package metadata (current_repodata.json): ...working... done

14 85.41 Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.

14 85.44 Collecting package metadata (repodata.json): ...working... done

14 489.9 Solving environment: ...working... done

14 619.6

14 619.6 ## Package Plan

14 619.6

14 619.6 environment location: /opt/conda/envs/xformers

14 619.6

14 619.6 added / updated specs:

14 619.6 - cudatoolkit=11.6

14 619.6 - pytorch=1.12.1

14 619.6

14 619.6

14 619.6 The following packages will be downloaded:

14 619.6

14 619.6 package | build

14 619.6 ---------------------------|-----------------

14 619.6 blas-1.0 | mkl 6 KB

14 619.6 ca-certificates-2022.12.7 | ha878542_0 143 KB conda-forge

14 619.6 certifi-2022.12.7 | pyhd8ed1ab_0 147 KB conda-forge

14 619.6 cudatoolkit-11.6.0 | hecad31d_10 821.2 MB conda-forge

14 619.6 intel-openmp-2022.1.0 | h9e868ea_3769 4.5 MB

14 619.6 mkl-2022.1.0 | hc2b9512_224 129.7 MB

14 619.6 pytorch-1.12.1 |py3.10_cuda11.6_cudnn8.3.2_0 1.20 GB pytorch

14 619.6 pytorch-mutex-1.0 | cuda 3 KB pytorch

14 619.6 typing_extensions-4.4.0 | pyha770c72_0 29 KB conda-forge

14 619.6 ------------------------------------------------------------

14 619.6 Total: 2.13 GB

14 619.6

14 619.6 The following NEW packages will be INSTALLED:

14 619.6

14 619.6 blas pkgs/main/linux-64::blas-1.0-mkl

14 619.6 cudatoolkit conda-forge/linux-64::cudatoolkit-11.6.0-hecad31d_10

14 619.6 intel-openmp pkgs/main/linux-64::intel-openmp-2022.1.0-h9e868ea_3769

14 619.6 mkl pkgs/main/linux-64::mkl-2022.1.0-hc2b9512_224

14 619.6 pytorch pytorch/linux-64::pytorch-1.12.1-py3.10_cuda11.6_cudnn8.3.2_0

14 619.6 pytorch-mutex pytorch/noarch::pytorch-mutex-1.0-cuda

14 619.6 typing_extensions conda-forge/noarch::typing_extensions-4.4.0-pyha770c72_0

14 619.6

14 619.6 The following packages will be UPDATED:

14 619.6

14 619.6 ca-certificates pkgs/main::ca-certificates-2022.10.11~ --> conda-forge::ca-certificates-2022.12.7-ha878542_0

14 619.6 certifi pkgs/main/linux-64::certifi-2022.9.24~ --> conda-forge/noarch::certifi-2022.12.7-pyhd8ed1ab_0

14 619.6

14 619.6

14 619.6 Proceed ([y]/n)?

14 619.6

14 619.6 Downloading and Extracting Packages

14 1110.5 CondaError: Downloaded bytes did not match Content-Length

14 1110.5 url: https://conda.anaconda.org/pytorch/linux-64/pytorch-1.12.1-py3.10_cuda11.6_cudnn8.3.2_0.tar.bz2

14 1110.5 target_path: /opt/conda/pkgs/pytorch-1.12.1-py3.10_cuda11.6_cudnn8.3.2_0.tar.bz2

14 1110.5 Content-Length: 1284916176

14 1110.5 downloaded bytes: 1100035059

14 1110.5

14 1110.5

14 1110.5

14 1126.1 ERROR conda.cli.main_run:execute(47): conda run /bin/bash -c conda install -c pytorch -c conda-forge cudatoolkit=11.6 pytorch=1.12.1 failed. (See above for error)

---

executor failed running [/opt/conda/bin/conda run --no-capture-output -n xformers /bin/bash -c conda install -c pytorch -c conda-forge cudatoolkit=11.6 pytorch=1.12.1]: exit code: 1

I understand that it's a download problem, but I'm not good at docker to be able to fix this problem.

Any suggestions?

[gadicc]

Hey!

I would have said just rerun the build command and it will retry again from the last successful (and cached) step, but as you say, you keep getting the same error.

Is it always on the same file, and the same number of bytes? Are you using a proxy?

[ormedo]

HI!

Thanks your your time supporting us. I think cloud be a bad file or networking issue. Upgrade to 11.7 version fix the problem

[ormedo]

Hi! Just another conflict.

In this case with the python version.

15 2074.9

15 2074.9 UnsatisfiableError: The following specifications were found

15 2074.9 to be incompatible with the existing python installation in your environment:

15 2074.9

15 2074.9 Specifications:

15 2074.9

15 2074.9 - six -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.8,<3.9.0a0|>=3.7,<3.8.0a0|>=3.9,<3.10.0a0|>=3.5,<3.6.0a0']

15 2074.9 - wheel -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.8,<3.9.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

15 2074.9 - xformers -> python[version='>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']

15 2074.9

15 2074.9 Your python: python=3.10

[gadicc]

Hey! Unfortunately xformers only has precompiled binaries for a very select list of package version combinations (I have some notes about this at the top of the Dockerfile). 11.7 won't work. You could try 11.3 though.

P.S. I don't know much about running on diffusers on an M1 beyond that it's possible. You may well need to search docker-diffusers-api codebase for anywhere I've written cuda and replace it with mps. I'll try fix this in a future release so this won't be necessary (you're the first person to try this :sweat_smile:) Please do report on your findings, would love to get this working for all M1 users!

[ormedo]

Its works con M1 with 11.3 :D but exited after a few seconds with no visible logs, at last with my knowledge :S

[ormedo]

[ormedo]

There are the logs inside container.

Traceback (most recent call last): File "/api/server.py", line 12, in user_src.init() File "/api/app.py", line 53, in init "device": torch.cuda.get_device_name(), File "/opt/conda/envs/xformers/lib/python3.10/site-packages/torch/cuda/init.py", line 329, in get_device_name return get_device_properties(device).name File "/opt/conda/envs/xformers/lib/python3.10/site-packages/torch/cuda/init.py", line 359, in get_device_properties _lazy_init() # will define _get_device_properties File "/opt/conda/envs/xformers/lib/python3.10/site-packages/torch/cuda/init.py", line 211, in _lazy_init raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled ERROR conda.cli.main_run:execute(47): conda run /bin/bash -c python3 -u server.py failed. (See above for error) libc10_cuda.so: cannot open shared object file: No such file or directory WARNING: libc10_cuda.so: cannot open shared object file: No such file or directory Need to compile C++ extensions to get sparse attention support. Please run python setup.py build develop

[gadicc]

Hey, thanks! Logs make it much easier to understand what's going on.

So yeah, as I suspected, unfortunately we're going to have to look for any code that references nvidia's cuda and remove it if it's not needed, or replace it with mps where possible, to work on Apple M1.

I would really love to make docker-diffusers-api work out the box with M1, but it's going to be quite a while until I'll have the time to be actively involved here :(

In the meantime, the line in question can be removed entirely (app.py line 53: device: torch.cuda...). And you'll need to search through all the files for any other mention of "cuda" and replace it with "mps" (especially anything like .to("cuda"), device="cuda", or anything like that).

Again, I wish I could help more, and look into automatically detecting the right GPU, but I just don't have time at the moment, and really am not sure when I will :( But please keep this issue open, please keep us updated with your progress, and I will take a more active role here when I can. I'll also be available to answer questions to the best of my ability (but I really have zero experience with Apple, unfortunately).

[gadicc]

And for future reference:

• https://discuss.pytorch.org/t/how-to-check-mps-availability/152015
• https://discuss.pytorch.org/t/could-it-possible-that-mps-is-made-as-easy-as-cuda/153882/2

[ormedo]

I Understand. I want to test before go on production in Banana's enviroment. But 1 Click installation goes sweet!

[gadicc]

Oh, awesome! That's great. Thanks for reporting back about that.. at least you can still play in the meantime :)

I should have a chance to look at this next week... if we're lucky, it will all just work afterwards. Otherwise it will take a lot longer :sweat_smile: Do you know any good places to rent M1's online? I think one of the companies I've used before has them, I'll try to remember :sweat_smile:

[ormedo]

AWS allow M1 Mac Mini instances if I remember well

[gadicc]

Oh great, thanks!

More future ref stuff for me...

https://pytorch.org/docs/stable/notes/mps.html

https://chrisdare.medium.com/running-pytorch-on-apple-silicon-m1-gpus-a8bb6f680b02