[Feature Request]: Upgrade PyTorch to 2.1.2 for Macs

[viking1304]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

New pytorch release 2.1.2 brings some bug fixes and optimizations for Macs.

With this version and the command line below, the generation time on M1 is about 20-25% faster. export COMMANDLINE_ARGS="--skip-torch-cuda-test --opt-sub-quad-attention --upcast-sampling --no-half-vae --medvram-sdxl --use-cpu interrogate"

Some people with M2 Macs said that generation time was reduced even more.

Proposed workflow

Replace

export COMMANDLINE_ARGS="--skip-torch-cuda-test --upcast-sampling --no-half-vae --use-cpu interrogate"
export TORCH_COMMAND="pip install torch==2.0.1 torchvision==0.15.2"

with

export COMMANDLINE_ARGS="--skip-torch-cuda-test --opt-sub-quad-attention --upcast-sampling --no-half-vae --medvram-sdxl --use-cpu interrogate"
export TORCH_COMMAND="pip install torch==2.1.2 torchvision==0.16.2"

in webui-macos-env.sh

Additional information

Most important Mac-related fixes in 2.1.2:

Fix crashes for float16 empty tensors (https://github.com/pytorch/pytorch/pull/115183) Fix MPS memory corruption when working with tensor slices (https://github.com/pytorch/pytorch/pull/114838) Fix crashes during Conv backward pass on MPS devices (https://github.com/pytorch/pytorch/pull/113398) Partially fix nn.Linear behavior on AArch64 platform (https://github.com/pytorch/pytorch/pull/110150)

I am at your disposal for any kind of tests on M1 MacBook Pro and iMac 2019 (Intel) with Radeon 580X 8GB

[ywj5782]

Need to upgrade on this basis

Input instructions pip install -U pip pip install -U httpcore

2： cuda 12.1 version pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu121

[viking1304]

I clearly wrote that this is a suggestion to the A1111 team for Mac computers only.

Please keep this thread clean of PyTorch upgrade comments related to other platforms.

[OilyBoHunk]

Thank you! SDXL time for 1 image (same prompt) on MAC M1 went from 2m02s to 1m31s.

[imruxin]

export TORCH_COMMAND="pip install torch==2.1.2 torchvision==0.16.2"

new is

export TORCH_COMMAND="pip install torch==2.2.1 torchvision==0.17.1"

[viking1304]

new is export TORCH_COMMAND="pip install torch==2.2.1 torchvision==0.17.1"

You are absolutely right.

I honestly forgot about this suggestion, since I am using my install script that always use the latest stable version of torch, whatever it is at that moment.

[viking1304]

Updating macOS to 14.4 breaks Euler A and some other samplers in A1111 if it uses Torch 2.2.0 or newer! More info

Reverting torch back to 2.1.2 solves the problem.

[galekseev]

Updating macOS to 14.4 breaks Euler A and some other samplers in A1111 if it uses Torch 2.2.0 or newer! More info

Reverting torch back to 2.1.2 solves the problem.

Thanks, it fixed Euler and Euler A samplers.

But reverting back to 2.1.2 on macOS 14.4 broke adetailer. It started to show " modules.devices.NansException: A tensor with all NaNs was produced in Unet" error, which I managed to fix with --no-half flag. But I wonder why it worked well with torch 2.0 and 2.2 without this flag and doesn't work with torch 2.1 because the solution with flag is a pain due to increased generation time.