Open 631068264 opened 3 months ago
Here is what I did to make CLIP interrogation work on my M3 Mac:
--no-half
flag prevents the system from using fp16. So, following the instruction in the Mac file, go to this file: https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/1c0a0c4c26f78c32095ebc7f8af82f5c04fca8c0/webui-user.sh#L13export COMMANDLINE_ARGS="--skip-torch-cuda-test --upcast-sampling --no-half-vae --use-cpu interrogate --no-half"
This makes the interrogation work, and SD1.5 models generate images fine
However, on my 18GB M3 Mac, loading my favorite SDXL model causes the Python memory usage to increase from the usual 10GB to over 20GB, halting image generation. To use the SDXL model again, I have to remove the --no-half
flag.
From my understanding, the M3 Mac uses a shared memory strategy, allowing both regular RAM and VRAM to use the same 18GB of RAM. However, when total usage exceeds 18GB, the system starts writing data to disk and swapping data between disk and RAM. This process is expensive and, since the SDXL Python process exceeds the 18GB supported by my Mac, even if the request is received, the swapping makes image generation impossible to complete in a reasonable time.
It's likely that the A1111 developers can fix this issue easily by making the interrogation use --no-half
by default in the Mac environment, regardless of the flag. This change can be updated in this file:
https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/1c0a0c4c26f78c32095ebc7f8af82f5c04fca8c0/modules/interrogate.py#L122.
Checklist
What happened?
Similar error on v1.9.3 on M3 Mac when use clip
error
Steps to reproduce the problem
Use image to image , and the use clip model for an image, get error
What should have happened?
get the prompt for image
What browsers do you use to access the UI ?
Google Chrome
Sysinfo
sysinfo-2024-05-14-03-07.json
Console logs
Additional information
No response