RuntimeError: CUDA error: operation not supported

[v4r4rth]

I am unable to get ComfyUI running on my GPU - it has 16GB VRAM and I am getting the following error:

C:\ComfyUI>python main.py Traceback (most recent call last): File "C:\ComfyUI\main.py", line 74, in import execution File "C:\ComfyUI\execution.py", line 12, in import nodes File "C:\ComfyUI\nodes.py", line 20, in import comfy.diffusers_load File "C:\ComfyUI\comfy\diffusers_load.py", line 4, in import comfy.sd File "C:\ComfyUI\comfy\sd.py", line 5, in from comfy import model_management File "C:\ComfyUI\comfy\model_management.py", line 114, in total_vram = get_total_memory(get_torch_device()) / (1024 * 1024) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\comfy\model_management.py", line 105, in get_totalmemory , mem_total_cuda = torch.cuda.mem_get_info(dev) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\uat\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\torch\cuda\memory.py", line 663, in mem_get_info return torch.cuda.cudart().cudaMemGetInfo(device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: CUDA error: operation not supported CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Any assistance? I tried changing the torch and cuda versions, reverted the changes and still not able to get it running...

[NeedsMoar]

Which GPU is it?

[v4r4rth]

It is an A40 GPU, 16GB Slice - I solved the issue by adding --disable-cuda-malloc to the commandline arguments, working perfectly now!

[XReyRobert]

had to do the same on a P6000, thanks

[jackyin68]

But this parameter is not supported now. How to deal with this problem？ main.py [-h] [--listen [IP]] [--port PORT] [--enable-cors-header [ORIGIN]] [--max-upload-size MAX_UPLOAD_SIZE] [--extra-model-paths-config PATH [PATH ...]] [--output-directory OUTPUT_DIRECTORY] [--temp-directory TEMP_DIRECTORY] [--input-directory INPUT_DIRECTORY] [--auto-launch] [--disable-auto-launch] [--cuda-device DEVICE_ID] [--cuda-malloc | --disable-cuda-malloc] [--dont-upcast-attention] [--force-fp32 | --force-fp16] [--bf16-unet | --fp16-unet | --fp8_e4m3fn-unet | --fp8_e5m2-unet] [--fp16-vae | --fp32-vae | --bf16-vae] [--fp8_e4m3fn-text-enc | --fp8_e5m2-text-enc | --fp16-text-enc | --fp32-text-enc] [--directml [DIRECTML_DEVICE]] [--disable-ipex-optimize] [--preview-method [none,auto,latent2rgb,taesd]] [--use-split-cross-attention | --use-quad-cross-attention | --use-pytorch-cross-attention] [--disable-xformers] [--gpu-only | --highvram | --normalvram | --lowvram | --novram | --cpu] [--disable-smart-memory]
[--deterministic] [--dont-print-server] [--quick-test-for-ci] [--windows-standalone-build] [--disable-metadata] main.py: error: unrecognized arguments: --disable-cudamalloc

[v4r4rth]

But this parameter is not supported now. How to deal with this problem？

main.py [-h] [--listen [IP]] [--port PORT] [--enable-cors-header [ORIGIN]] [--max-upload-size MAX_UPLOAD_SIZE] [--extra-model-paths-config PATH [PATH ...]] [--output-directory OUTPUT_DIRECTORY] [--temp-directory TEMP_DIRECTORY] [--input-directory INPUT_DIRECTORY] [--auto-launch] [--disable-auto-launch] [--cuda-device DEVICE_ID] [--cuda-malloc | --disable-cuda-malloc] [--dont-upcast-attention] [--force-fp32 | --force-fp16] [--bf16-unet | --fp16-unet | --fp8_e4m3fn-unet | --fp8_e5m2-unet] [--fp16-vae | --fp32-vae | --bf16-vae] [--fp8_e4m3fn-text-enc | --fp8_e5m2-text-enc | --fp16-text-enc | --fp32-text-enc] [--directml [DIRECTML_DEVICE]] [--disable-ipex-optimize] [--preview-method [none,auto,latent2rgb,taesd]] [--use-split-cross-attention | --use-quad-cross-attention | --use-pytorch-cross-attention] [--disable-xformers] [--gpu-only | --highvram | --normalvram | --lowvram | --novram | --cpu] [--disable-smart-memory] [--deterministic] [--dont-print-server] [--quick-test-for-ci] [--windows-standalone-build] [--disable-metadata] main.py: error: unrecognized arguments: --disable-cudamalloc

You have a typo: --disable-cuda-malloc instead of disable-cudamalloc

[jackyin68]

Fine, many thanks to you!

[denyshubh]

After changing my .\run_nvidia_gpu.bat to below code fixed my issue.

.\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build --disable-cuda-malloc
pause

[luojin]

thx