-
Right now is table to avoid redundant reads.
By splitting data on memory we could provide more fine-granular parallelism, i.e. per column, while still avoiding redundant reads.
-
**Describe the problem**
Part of https://github.com/cockroachdb/cockroach/issues/110432 - we need to add a node level buffered stream sender to buffer events at the node level. We should also refac…
-
We are currently trying to apply torchtitan to MoE models. MoE models require using grouped_gemm https://github.com/fanshiqing/grouped_gemm. GroupedGemm ops basically follow the same rule as in Column…
-
Hello. I am curious if it would be possible to implement Megatron-style sequence parallelism in the repository.
Sequence parallelism is important for reducing activation memory, which is difficult to…
-
It takes a lot of time to run it now. Thanks!
-
**Describe the bug**
Just like this PR: https://github.com/microsoft/DeepSpeed/pull/5259 , ZeRO optimizer also needs to be fixed:
1. partition logic of expert params.
3. average_tensor used in …
-
### OpenTofu Version
```shell
1.6.2
```
### Use Cases
Callling API limited providers in the same module/run.
I.e. mixing grafana and aws or status page.
### Attempted Solutions
Using terragrunt…
-
Hi admin, if I managed to run 2 algorithms simultaneously at one time...
Does it considered as parallelism for the extra features?
I can provide the video of the program.
-
Modern computers have multiple cores, but we're only using two of them (one for UI, one for AI). The AI can (and should) use multiple cores if available!
-
### 🚀 The feature, motivation and pitch
# introduction
Zero Bubble support for pipe see:https://arxiv.org/abs/2401.10241
# require
when we want to implement this feature. we need to solve the …