search

rhymes-ai / Aria

Codebase for Aria - an Open Multimodal Native MoE
Apache License 2.0
375 stars 20 forks source link

vllm multi gpu #24

Open matbee-eth opened 1 day ago

matbee-eth commented 1 day ago

Any help on getting multi gpu support running? vLLM fails to load with tensor_parallel_size=2

xffxff commented 1 day ago

@matbee-eth Can you include more details about the error to help us debug it?