search

StartHua / Comfyui_CXH_joy_caption

Recommended based on comfyui node pictures:Joy_caption + MiniCPMv2_6-prompt-generator + florence2
Apache License 2.0
446 stars 26 forks source link

Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit the quantized model. #74

Open yatoubusha opened 2 months ago

yatoubusha commented 2 months ago

Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit the quantized model. If you want to dispatch the model on the CPU or the disk while keeping these modules in 32-bit, you need to set load_in_8bit_fp32_cpu_offload=True and pass a custom device_map to from_pretrained. Check https://huggingface.co/docs/transformers/main/en/main_classes/quantization#offload-between-cpu-and-gpu for more details.

how to solve this error, i have 24G gpu, my transformers==4.44.2 image

tamas-hortobagyi-deltatre commented 2 months ago

The same happens here. Could it be Windows related?