-
Hey,
We just published a pytest plugin to automate running a test suite in parallel since this isn't possible with pytest-xdist.
https://pypi.org/project/pytest-freethreaded/
The pattern is loosely…
-
I'm using code similar to the 8-way batch data parallelism example here: https://jax.readthedocs.io/en/latest/notebooks/Distributed_arrays_and_automatic_parallelization.html#way-batch-data-parallelism…
-
### 🚀 The feature, motivation and pitch
I want to add 2D tensor parallelism and expert parallelism in vLLM. Which parts of the code should I modify?
### Alternatives
_No response_
### Additional c…
-
There are times when wheels could be build in parallel. We may see some performance improvements if we support that.
-
I think we can improve the processing speed by a lot by using the joblib package: https://joblib.readthedocs.io/en/latest/parallel.html
As we know, huffman decoding is hard to parallelise but we ca…
-
Current parallel strategy assign different files in a directory to different workers.
There are many situations where this may incur load unbalancing, for example, when the input files are irregula…
-
Hi,
Let's say, I have a slurm cluster that contains 100 nodes, each node has 100 cores. Assuming I have 10000 tasks.
This is my current code:
```
dist_executor = SlurmPipelineExecutor(
…
-
### Proposal to improve performance
_No response_
### Report of performance regression
_No response_
### Misc discussion on performance
Hi,
Thank you for your contribution to the LLM community…
-
Greetings!
In relation to this #54 I am running into a locked ref issue.
` " ! [remote rejected] @test-repo/pkg-b-v2.0.0 -> @test-repo/pkg-b-v2.0.0 (cannot lock ref 'refs/tags/@test-repo/pkg-b…
-
### 🚀 The feature, motivation and pitch
We can combine pipeline parallelism with speculative decoding to get latency reductions, especially when serving Llama 405b over two nodes.
The speculative …