-
(edit: I'm using python 3.10.8 and both dask and distributed are version 2022.3.1)
I have several issues running this code:
Downsampling + rechunking
```python
import dask.array as da
impo…
d-v-b updated
10 months ago
-
### System Info
```Shell
- `Accelerate` version: 0.34.2
- Platform: Linux-5.15.0-1071-aws-x86_64-with-glibc2.31
- `accelerate` bash location: /home/zeus/miniconda3/envs/cloudspace/bin/accelerate
-…
-
Several tests in test_shuffle.py are very flaky.
If I change `.github/workflows/tests.yaml` as follows, to rerun the tests 20 times (ci1 + not ci1) per environment:
```
pytest distribut…
-
Next Demo Day: December 5th
---
See what the Dask community has been up to, or share some Dask work of your own. Demos are short and informal (~5-10 minutes). Have something you'd like to share? L…
-
Context :- I am trying to run distributed training on 2 A-100 gpus with 40GB of VRAM. The batch size is 3 and gradient accumulation=1. I have attached the config file below for more details and the er…
-
### Search before asking
- [X] I have searched the Ultralytics YOLO [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussion…
-
🧠 rewind
✔️ parquet file[🔗](https://github.com/seoyeong200/LeetCode/issues/15#issuecomment-2412094885)
\- with spark read performance
🀄️ b-tree, b+tree, isolation level [🔗](https://github.com/s…
-
I am following the [getting started](https://epfllm.github.io/Megatron-LLM/guide/getting_started.html) guide with mistal-7B model.
- I am able to (1) convert `mistralai/Mistral-7B-v0.1` and (2) …
-
When there are object columns with mixed data types, the arrow backend cannot handle this.
The internal error that is raised in this example is
```python-traceback
ArrowInvalid: ('cannot mix li…
-
## TL:DR
We currently allow for stealing of `split-shuffle` tasks. That's a really bad idea...
## Intro
Our distributed shuffle algorithm currently employs three kinds of tasks which are repe…