-
**Is your feature request related to a problem? Please describe.**
Tried to run custom 40B model, whose weights can be loaded with 2 80GB GPU's VRAM.
lmcache is able to load small models with in sin…
-
-
Python 3.11.10 | packaged by conda-forge | (main, Sep 22 2024, 14:10:38) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from vllm import SamplingPar…
-
你好，我运行`sh tools/dist_train.sh projects/configs/co_dino_vit/co_dino_5scale_vit_large_coco.py 1`时，会如下错误，我看之前也有人报这个错，请问这个问题如何解决？谢谢
:
File "/workspace/D-FINE/src/nn/back…
-
@ericneiva
The moment fitting integration is not currently working on distributed.
I have done some progress in my fork [distributed_moment_fitting](https://github.com/pmartorell/GridapEmbedde…
-
We just explored osctrl-admin and found that we can add a custom tag to each node/device. However, after added the custom tag, we cannot run a distributed query based on this tag. It would be great he…
-
### Your current environment
kuberay，vllm 0.4.0
L40 GPU server *2， each one with L40*8, CX6 IB card 200G*2
### How would you like to use vllm
I plan to use KubeRay to implement multi-node distri…