device_map provided does not give any device

ShareGPT4Omni / ShareGPT4V

[ECCV 2024] ShareGPT4V: Improving Large Multi-modal Models with Better Captions

https://sharegpt4v.github.io/

124 stars 4 forks source link

Closed zimenglan-sysu-512 closed 3 months ago

zimenglan-sysu-512 commented 3 months ago

when batch infer to produce captions using eight GPUs, encounter the problem:

ValueError: The device_map provided does not give any device for the following parameters

zimenglan-sysu-512 commented 3 months ago

another question, set batch_size=1, the GPU occupies 31G, does it make sense?