-
I wonder if I might be doing something wrong, but it appears that all of my model evaluations are occurring in serial when I believe they should be occurring in parallel.
If I define my `allocation…
-
### Checklist
- [x] 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.…
-
### ⚠️ Please check that this feature request hasn't been suggested before.
- [X] I searched previous [Ideas in Discussions](https://github.com/axolotl-ai-cloud/axolotl/discussions/categories/ideas) …
-
On `PP + FSDP` and `PP + TP + FSDP`:
- Is there any documentation on how these different parallelisms compose?
- What are the largest training runs these strategies have been tested on?
- Are there…
-
### Your current environment
The output of `python collect_env.py` on Node 0 (head node)
```text
root@awsA10G:/workspace# python collect_env.py
/usr/local/lib/python3.10/dist-packages/vllm/co…
-
i have a collection with ~2000 documents, each of ~10k-20k bytes.. so maybe 10-40 megabytes.
when i submit a single get() of all documents in this collection via the node.js SDK, i get great respo…
-
Hi all,
the provided script `train_pyramid_flow.sh` does not set the flag `use_sequence_parallel`. In that case, what is the purpose of using `VIDEO_SYNC_GROUP=8`? Why we want all workers to use t…
-
### 🚀 The feature, motivation and pitch
I noticed that the current speculative mode does not support tp from this link (https://docs.vllm.ai/en/stable/models/spec_decode.html).
However, not suppo…
-
The general question is, does mamba-ssm currently support sequence parallelism in the mixer?
I noticed that Section 8.2 in the paper of Mamba2 proposes a potential way to split activation among mu…
-
### Your current environment
This issue is easy to reproduce. In AWS:
1) Spin up EC2
2) Use the Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.3.1 (Ubuntu 20.04)
3) Select g5.12xlarge (which co…