renovate[bot]
commented
1 month ago Branch automerge failure
This PR was configured for branch automerge. However, this is not possible, so it has been raised as a PR instead.
- Branch has one or more failed status checks
This PR contains the following updates:
==2.20.0->==2.21.0Release Notes
huggingface/datasets (datasets)
### [`v2.21.0`](https://togithub.com/huggingface/datasets/releases/tag/2.21.0) [Compare Source](https://togithub.com/huggingface/datasets/compare/2.20.0...2.21.0) #### Features - Support pyarrow large_list by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7019](https://togithub.com/huggingface/datasets/pull/7019) - Support Polars round trip: ```python import polars as pl from datasets import Dataset df1 = pl.from_dict({"col_1": [[1, 2], [3, 4]]} df2 = Dataset.from_polars(df).to_polars() assert df1.equals(df2) ``` #### What's Changed - Use `HF_HUB_OFFLINE` instead of `HF_DATASETS_OFFLINE` by [@Wauplin](https://togithub.com/Wauplin) in [https://github.com/huggingface/datasets/pull/6968](https://togithub.com/huggingface/datasets/pull/6968) - packaging: Remove useless dependencies by [@daskol](https://togithub.com/daskol) in [https://github.com/huggingface/datasets/pull/6971](https://togithub.com/huggingface/datasets/pull/6971) - Fix resuming arrow format by [@lhoestq](https://togithub.com/lhoestq) in [https://github.com/huggingface/datasets/pull/6964](https://togithub.com/huggingface/datasets/pull/6964) - Fix webdataset pickling by [@lhoestq](https://togithub.com/lhoestq) in [https://github.com/huggingface/datasets/pull/6972](https://togithub.com/huggingface/datasets/pull/6972) - Set temporary numpy upper version < 2.0.0 to fix CI by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/6975](https://togithub.com/huggingface/datasets/pull/6975) - Fix regression for pandas < 2.0.0 in JSON loader by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/6978](https://togithub.com/huggingface/datasets/pull/6978) - Ensure compatibility with numpy 2.0.0 by [@KennethEnevoldsen](https://togithub.com/KennethEnevoldsen) in [https://github.com/huggingface/datasets/pull/6976](https://togithub.com/huggingface/datasets/pull/6976) - Remove underlines between badges by [@novialriptide](https://togithub.com/novialriptide) in [https://github.com/huggingface/datasets/pull/6966](https://togithub.com/huggingface/datasets/pull/6966) - Update docs on trust_remote_code defaults to False by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/6981](https://togithub.com/huggingface/datasets/pull/6981) - Improve skip take shuffling and distributed by [@lhoestq](https://togithub.com/lhoestq) in [https://github.com/huggingface/datasets/pull/6965](https://togithub.com/huggingface/datasets/pull/6965) - Fix tests using hf-internal-testing/librispeech_asr_dummy by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/6998](https://togithub.com/huggingface/datasets/pull/6998) - Fix dump of bfloat16 torch tensor by [@lhoestq](https://togithub.com/lhoestq) in [https://github.com/huggingface/datasets/pull/7002](https://togithub.com/huggingface/datasets/pull/7002) - minor fix for bfloat16 by [@lhoestq](https://togithub.com/lhoestq) in [https://github.com/huggingface/datasets/pull/7003](https://togithub.com/huggingface/datasets/pull/7003) - Fix incorrect rank value in data splitting by [@yzhangcs](https://togithub.com/yzhangcs) in [https://github.com/huggingface/datasets/pull/6994](https://togithub.com/huggingface/datasets/pull/6994) - less script docs by [@lhoestq](https://togithub.com/lhoestq) in [https://github.com/huggingface/datasets/pull/6993](https://togithub.com/huggingface/datasets/pull/6993) - Fix CI by temporarily pinning ruff < 0.5.0 by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7007](https://togithub.com/huggingface/datasets/pull/7007) - Support ruff 0.5.0 in CI by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7009](https://togithub.com/huggingface/datasets/pull/7009) - Fix WebDatasets KeyError for user-defined Features when a field is missing in an example by [@ProGamerGov](https://togithub.com/ProGamerGov) in [https://github.com/huggingface/datasets/pull/7004](https://togithub.com/huggingface/datasets/pull/7004) - \[Streaming] retry on requests errors by [@lhoestq](https://togithub.com/lhoestq) in [https://github.com/huggingface/datasets/pull/6963](https://togithub.com/huggingface/datasets/pull/6963) - Re-enable raising error from huggingface-hub FutureWarning in CI by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7011](https://togithub.com/huggingface/datasets/pull/7011) - Skip faiss tests on Windows to avoid running CI for 360 minutes by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7014](https://togithub.com/huggingface/datasets/pull/7014) - Support fsspec 2024.6.1 by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7017](https://togithub.com/huggingface/datasets/pull/7017) - Persist IterableDataset epoch in workers by [@lhoestq](https://togithub.com/lhoestq) in [https://github.com/huggingface/datasets/pull/6710](https://togithub.com/huggingface/datasets/pull/6710) - Fix casting list array to fixed size list by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7021](https://togithub.com/huggingface/datasets/pull/7021) - Remove dead code for pyarrow < 15.0.0 by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7023](https://togithub.com/huggingface/datasets/pull/7023) - Fix check_library_imports by [@lhoestq](https://togithub.com/lhoestq) in [https://github.com/huggingface/datasets/pull/7026](https://togithub.com/huggingface/datasets/pull/7026) - Missing line from previous pr by [@lhoestq](https://togithub.com/lhoestq) in [https://github.com/huggingface/datasets/pull/7027](https://togithub.com/huggingface/datasets/pull/7027) - Fix ci by [@lhoestq](https://togithub.com/lhoestq) in [https://github.com/huggingface/datasets/pull/7028](https://togithub.com/huggingface/datasets/pull/7028) - Add decorator as explicit test dependency by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7043](https://togithub.com/huggingface/datasets/pull/7043) - Mark tests that require librosa by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7044](https://togithub.com/huggingface/datasets/pull/7044) - Unblock NumPy 2.0 by [@NeilGirdhar](https://togithub.com/NeilGirdhar) in [https://github.com/huggingface/datasets/pull/6991](https://togithub.com/huggingface/datasets/pull/6991) - Fix tensorflow min version depending on Python version by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7045](https://togithub.com/huggingface/datasets/pull/7045) - Support librosa and numpy 2.0 for Python 3.10 by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7046](https://togithub.com/huggingface/datasets/pull/7046) - add checkpoint and resume title in docs by [@lhoestq](https://togithub.com/lhoestq) in [https://github.com/huggingface/datasets/pull/7050](https://togithub.com/huggingface/datasets/pull/7050) - Update load_hub.mdx by [@severo](https://togithub.com/severo) in [https://github.com/huggingface/datasets/pull/7057](https://togithub.com/huggingface/datasets/pull/7057) - Add batching to IterableDataset by [@lappemic](https://togithub.com/lappemic) in [https://github.com/huggingface/datasets/pull/7054](https://togithub.com/huggingface/datasets/pull/7054) - Avoid calling http_head for non-HTTP URLs by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7062](https://togithub.com/huggingface/datasets/pull/7062) - Fix load_dataset for data_files with protocols other than HF by [@matstrand](https://togithub.com/matstrand) in [https://github.com/huggingface/datasets/pull/6862](https://togithub.com/huggingface/datasets/pull/6862) - Add batch method to Dataset class by [@lappemic](https://togithub.com/lappemic) in [https://github.com/huggingface/datasets/pull/7064](https://togithub.com/huggingface/datasets/pull/7064) - Fix doc generation when NamedSplit is used as parameter default value by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7036](https://togithub.com/huggingface/datasets/pull/7036) - Fix CI by temporarily marking test_convert_to_parquet as expected to fail by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7074](https://togithub.com/huggingface/datasets/pull/7074) - add split argument to Generator by [@piercus](https://togithub.com/piercus) in [https://github.com/huggingface/datasets/pull/7015](https://togithub.com/huggingface/datasets/pull/7015) - Update required soxr version from pre-release to release by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7075](https://togithub.com/huggingface/datasets/pull/7075) - Fix CI test_convert_to_parquet by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7078](https://togithub.com/huggingface/datasets/pull/7078) - Fix prepare_single_hop_path_and_storage_options by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7068](https://togithub.com/huggingface/datasets/pull/7068) - Set load_from_disk path type as PathLike by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7081](https://togithub.com/huggingface/datasets/pull/7081) - Fix push_to_hub by not calling create_branch if branch exists by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7069](https://togithub.com/huggingface/datasets/pull/7069) - feat: support non streamable arrow file binary format by [@kmehant](https://togithub.com/kmehant) in [https://github.com/huggingface/datasets/pull/7025](https://togithub.com/huggingface/datasets/pull/7025) - Support HTTP authentication in non-streaming mode by [@albertvillanova](https://togithub.com/albertvillanova) in [https://github.com/huggingface/datasets/pull/7082](https://togithub.com/huggingface/datasets/pull/7082) - chore: fix typos in docs by [@hattizai](https://togithub.com/hattizai) in [https://github.com/huggingface/datasets/pull/7034](https://togithub.com/huggingface/datasets/pull/7034) - Fix CI for metrics by [@albertvillanova](https://togithub.com/albertvillanova) in https://github.com/huggingface/datasets/commit/83e5c05fd38a4a37b5e6d5d7c0cfa73d76f1b220 #### New Contributors - [@novialriptide](https://togithub.com/novialriptide) made their first contribution in [https://github.com/huggingface/datasets/pull/6966](https://togithub.com/huggingface/datasets/pull/6966) - [@yzhangcs](https://togithub.com/yzhangcs) made their first contribution in [https://github.com/huggingface/datasets/pull/6994](https://togithub.com/huggingface/datasets/pull/6994) - [@ProGamerGov](https://togithub.com/ProGamerGov) made their first contribution in [https://github.com/huggingface/datasets/pull/7004](https://togithub.com/huggingface/datasets/pull/7004) - [@NeilGirdhar](https://togithub.com/NeilGirdhar) made their first contribution in [https://github.com/huggingface/datasets/pull/6991](https://togithub.com/huggingface/datasets/pull/6991) - [@matstrand](https://togithub.com/matstrand) made their first contribution in [https://github.com/huggingface/datasets/pull/6862](https://togithub.com/huggingface/datasets/pull/6862) - [@lappemic](https://togithub.com/lappemic) made their first contribution in [https://github.com/huggingface/datasets/pull/7054](https://togithub.com/huggingface/datasets/pull/7054) - [@piercus](https://togithub.com/piercus) made their first contribution in [https://github.com/huggingface/datasets/pull/7015](https://togithub.com/huggingface/datasets/pull/7015) - [@kmehant](https://togithub.com/kmehant) made their first contribution in [https://github.com/huggingface/datasets/pull/7025](https://togithub.com/huggingface/datasets/pull/7025) - [@hattizai](https://togithub.com/hattizai) made their first contribution in [https://github.com/huggingface/datasets/pull/7034](https://togithub.com/huggingface/datasets/pull/7034) **Full Changelog**: https://github.com/huggingface/datasets/compare/2.20.0...2.21.0Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Enabled.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.