-
### Describe the bug
**Description**
Providing a generator in an instantiation of IterableDataset.from_generator() fails with `TypeError: cannot pickle 'generator' object` when the generator argumen…
-
Hi everyone!
I have just published this project on GitHub: https://github.com/davidmartinrius/speech-dataset-generator/
Now you can create datasets automatically with any audio or lists of audi…
-
This happened once before and got fixed: https://github.com/EleutherAI/lm-evaluation-harness/issues/898
But now it seems not working again with same error, at least on my end.
```bash
File "/…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
```
Generating train split: 0 examples [00:00, ? examples/s]Failed to convert pandas Da[62/1867]
…
-
Did you try to use transfer learning for generator training, ie using FFHQ generator weights as initial state for LSUN datasets training?
-
### Describe the bug
Hi, I am preprocessing a large custom dataset with numpy arrays. I am running into this TypeError during writing in a dataset.map() function. I've tried decreasing writer batch s…
-
### Feature request
Similar to non-iterable datasets I would like functionality to group batches by similar length inputs.
Example of how this could be implemented is using a buffer to preload ba…
-
/root/miniconda3/envs/ISNet-cu118/lib/python3.8/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
warnings.…
-
If I have more tasks than 1k, datatrove splits it into multiple job arrays 1k-each.
the first job array of 1k runs fine, the subsequent ones all fail
```
0: 2024-07-04 23:59:34.496 | ERROR |…
-
Certainly! In Python, a generator is a type of iterable, like a list or a tuple, but unlike lists, generators do not store all their values in memory at once. Instead, they generate values on-the-fly …