-
The following pattern is common is user code and our examples:
```
def shard(pytree, n_devices):
def _shard_array(array):
return array.reshape((n_devices, -1) + array.shape[1:])
return ja…
-
### Describe the bug
1. https://github.com/opensearch-project/OpenSearch/issues/13207
2. https://github.com/opensearch-project/OpenSearch/issues/12788
3. https://github.com/opensearch-project/OpenS…
-
I'd like to introduce a table sampling feature. Specifically, it would be beneficial if we could perform sampling with a fixed shard number. This would allow us to calculate the distinct count of data…
-
-
https://buildkite.com/redpanda/vtools/builds/15137
https://buildkite.com/redpanda/vtools/builds/15138
```
Module: rptest.scale_tests.many_partitions_test
Class: ManyPartitionsTest
Method: test_many_…
-
Teodorikus, a member of the Omega SMP has reported resurrection shards to not work. They do not own a GitHub account, so I will be adressing this issue for them.
Expected behavior:
-on death-ban, …
-
This happens when running the new nayduck test in https://github.com/near/nearcore/pull/11676:
1) Start nodes with memtries enabled.
2) Restart nodes with memtries disabled.
3) Restart nodes with m…
-
GC is designed around the invariant that there is a set of blocks from tail to head,
each block stores a set of changes, and storage supports adding to head and removing from tail.
To support this i…
-
### Describe the feature
**Describe the feature:**
`Training in a large scale env with multiple DP group strategy will load huge data load throughtput. Ex: Training LLAMA2-70B (the checkpoint is 70…
-
When interacting with shards directly its nice to have kubectl, ws, etc commands in the image.