Bump datasets from 2.20.0 to 2.21.0

[dependabot[bot]]

Bumps datasets from 2.20.0 to 2.21.0.

Release notes

Sourced from datasets's releases.

2.21.0

Features

• Support pyarrow large_list by @​albertvillanova in huggingface/datasets#7019
  • Support Polars round trip:

    import polars as pl
    from datasets import Dataset
    df1 = pl.from_dict({"col_1": [[1, 2], [3, 4]]}
    df2 = Dataset.from_polars(df).to_polars()
    assert df1.equals(df2)
    

What's Changed

• Use HF_HUB_OFFLINE instead of HF_DATASETS_OFFLINE by @​Wauplin in huggingface/datasets#6968
• packaging: Remove useless dependencies by @​daskol in huggingface/datasets#6971
• Fix resuming arrow format by @​lhoestq in huggingface/datasets#6964
• Fix webdataset pickling by @​lhoestq in huggingface/datasets#6972
• Set temporary numpy upper version < 2.0.0 to fix CI by @​albertvillanova in huggingface/datasets#6975
• Fix regression for pandas < 2.0.0 in JSON loader by @​albertvillanova in huggingface/datasets#6978
• Ensure compatibility with numpy 2.0.0 by @​KennethEnevoldsen in huggingface/datasets#6976
• Remove underlines between badges by @​novialriptide in huggingface/datasets#6966
• Update docs on trust_remote_code defaults to False by @​albertvillanova in huggingface/datasets#6981
• Improve skip take shuffling and distributed by @​lhoestq in huggingface/datasets#6965
• Fix tests using hf-internal-testing/librispeech_asr_dummy by @​albertvillanova in huggingface/datasets#6998
• Fix dump of bfloat16 torch tensor by @​lhoestq in huggingface/datasets#7002
• minor fix for bfloat16 by @​lhoestq in huggingface/datasets#7003
• Fix incorrect rank value in data splitting by @​yzhangcs in huggingface/datasets#6994
• less script docs by @​lhoestq in huggingface/datasets#6993
• Fix CI by temporarily pinning ruff < 0.5.0 by @​albertvillanova in huggingface/datasets#7007
• Support ruff 0.5.0 in CI by @​albertvillanova in huggingface/datasets#7009
• Fix WebDatasets KeyError for user-defined Features when a field is missing in an example by @​ProGamerGov in huggingface/datasets#7004
• [Streaming] retry on requests errors by @​lhoestq in huggingface/datasets#6963
• Re-enable raising error from huggingface-hub FutureWarning in CI by @​albertvillanova in huggingface/datasets#7011
• Skip faiss tests on Windows to avoid running CI for 360 minutes by @​albertvillanova in huggingface/datasets#7014
• Support fsspec 2024.6.1 by @​albertvillanova in huggingface/datasets#7017
• Persist IterableDataset epoch in workers by @​lhoestq in huggingface/datasets#6710
• Fix casting list array to fixed size list by @​albertvillanova in huggingface/datasets#7021
• Remove dead code for pyarrow < 15.0.0 by @​albertvillanova in huggingface/datasets#7023
• Fix check_library_imports by @​lhoestq in huggingface/datasets#7026
• Missing line from previous pr by @​lhoestq in huggingface/datasets#7027
• Fix ci by @​lhoestq in huggingface/datasets#7028
• Add decorator as explicit test dependency by @​albertvillanova in huggingface/datasets#7043
• Mark tests that require librosa by @​albertvillanova in huggingface/datasets#7044
• Unblock NumPy 2.0 by @​NeilGirdhar in huggingface/datasets#6991
• Fix tensorflow min version depending on Python version by @​albertvillanova in huggingface/datasets#7045
• Support librosa and numpy 2.0 for Python 3.10 by @​albertvillanova in huggingface/datasets#7046
• add checkpoint and resume title in docs by @​lhoestq in huggingface/datasets#7050
• Update load_hub.mdx by @​severo in huggingface/datasets#7057
• Add batching to IterableDataset by @​lappemic in huggingface/datasets#7054

... (truncated)

Commits

• a1b5a32 Release: 2.21.0 (#7098)
• 83e5c05 Fix CI for metrics
• af818af chore: fix typos in docs (#7034)
• be43e09 Support pyarrow large_list (#7019)
• e7094be Support HTTP authentication in non-streaming mode (#7082)
• 5735e6b feat: support non streamable arrow file binary format (#7025)
• 4d5e237 Fix push_to_hub by not calling create_branch if branch exists (#7069)
• 87e8ab3 Set load_from_disk path type as PathLike (#7081)
• 360d8c1 Fix prepare_single_hop_path_and_storage_options (#7068)
• 779f6d7 Fix CI test_convert_to_parquet (#7078)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)