-
Just wonder does the current PipelineStage API supports variable length input shapes like in Megatron? https://github.com/NVIDIA/Megatron-LM/blob/e33c8f78a35765d5aa37475a144da60e8a2349d1/megatron/core…
-
Hello,
We are using torchrec and the two types of parallelism in our system. To have a better understanding on the details and seek the best communication primitives for our code, I want to know so…
-
Argon2 and scrypt have a parallelism parameter, but it's [often not used](https://crypto.stackexchange.com/a/84085)/[recommended to stay at 1](https://github.com/Sc00bz/bscrypt?tab=readme-ov-file#sett…
-
It seems torchrec does not support the combination of data parallelism and row-wise parallelism for embedding. I want to know is there a plan on it? Or is row-wise parallelism efficient enough when it…
-
Hi,
I am curious if it would be possible to implement Megatron-style sequence parallelism in the repository.
Sequence parallelism is crucial for reducing activation memory, which is otherwise ch…
-
Currently, we already reached the GitHub action parallelism limit.
Consider adding other CI services to achieve better parallelism to speed up the CI process.
Circle CI looks good to me, as they s…
-
### System Info
latest TGI docker image
### Information
- [X] Docker
- [ ] The CLI directly
### Tasks
- [X] An officially supported command
- [ ] My own modifications
### Reproduction
1. Use …
-
Is there a plan that supports parallelism?
-
Hi,
Pipefusion - 'Considering DiT’s affinity to Large Language Models (LLMs), both tensor parallelism and sequence parallelism, which are commonly utilized for efficient inference in LLMs, can be a…
-
Does this natively support parallelism accross gpus?
Also feature request: natively perform flash attention please