We tried to launch the model worker on a machine with multiple RTX3090 GPUs, but we couldn't use this command python -m llava.serve.model_worker --host 0.0.0.0 --controller http://localhost:10000 --port 40000 --worker http://localhost:40000 --model-path microsoft/llava-med-v1.5-mistral-7b --multi-modal --num-gpus 2. But after running it we got an error: TypeError: LlavaMistralForCausalLM.__init__() got an unexpected keyword argument 'num_gpus'.
thedaffodil
commented
1 month ago
We tried to launch the model worker on a machine with multiple RTX3090 GPUs, but we couldn't use this command
python -m llava.serve.model_worker --host 0.0.0.0 --controller http://localhost:10000 --port 40000 --worker http://localhost:40000 --model-path microsoft/llava-med-v1.5-mistral-7b --multi-modal --num-gpus 2. But after running it we got an error:TypeError: LlavaMistralForCausalLM.__init__() got an unexpected keyword argument 'num_gpus'.