v0.25.0: Large uploads made simple + quality of life improvements
📂 Upload large folders
Uploading large models or datasets is challenging. We've already written some tips and tricks to facilitate the process but something was still missing. We are now glad to release the huggingface-cli upload-large-folder command. Consider it as a "please upload this no matter what, and be quick" command. Contrarily to huggingface-cli download, this new command is more opinionated and will split the upload into several commits. Multiple workers are started locally to hash, pre-upload and commit the files in a way that is resumable, resilient to connection errors, and optimized against rate limits. This feature has already been stress tested by the community over the last months to make it as easy and convenient to use as possible.
.. warning:: Driver support for MongoDB 3.6 reached end of life in April 2024.
PyMongo 4.9 will be the last release to support MongoDB 3.6.
.. warning:: PyMongo 4.9 refactors a large portion of internal APIs to support the new asynchronous API beta.
As a result, versions of Motor older than 3.6 are not compatible with PyMongo 4.9.
Existing users of these versions must either upgrade to Motor 3.6 and PyMongo 4.9,
or cap their PyMongo version to < 4.9.
Any applications that use private APIs may also break as a result of these internal changes.
PyMongo 4.9 brings a number of improvements including:
Added support for MongoDB 8.0.
Added support for Python 3.13.
A new beta asynchronous API with full asyncio support.
This new asynchronous API is a work-in-progress that may change during the beta period before the full release.
Added support for In-Use Encryption range queries with MongoDB 8.0.
Added :attr:~pymongo.encryption.Algorithm.RANGE.
sparsity and trim_factor are now optional in :class:~pymongo.encryption_options.RangeOpts.
Added support for the "delegated" option for the KMIP master_key in
:meth:~pymongo.encryption.ClientEncryption.create_data_key.
pymongocrypt>=1.10 is now required for :ref:In-Use Encryption support.
Added :meth:~pymongo.cursor.Cursor.to_list to :class:~pymongo.cursor.Cursor,
:class:~pymongo.command_cursor.CommandCursor,
:class:~pymongo.asynchronous.cursor.AsyncCursor,
and :class:~pymongo.asynchronous.command_cursor.AsyncCommandCursor
as an asynchronous-friendly alternative to list(cursor).
Added :meth:~pymongo.mongo_client.MongoClient.bulk_write to :class:~pymongo.mongo_client.MongoClient
and :class:~pymongo.asynchronous.mongo_client.AsyncMongoClient,
enabling users to perform insert, update, and delete operations
against mixed namespaces in a minimized number of round trips.
Please see :doc:examples/client_bulk for more information.
Added support for the namespace parameter to the
:class:~pymongo.operations.InsertOne,
:class:~pymongo.operations.ReplaceOne,
:class:~pymongo.operations.UpdateOne,
:class:~pymongo.operations.UpdateMany,
:class:~pymongo.operations.DeleteOne, and
:class:~pymongo.operations.DeleteMany operations, so
they can be used in the new :meth:~pymongo.mongo_client.MongoClient.bulk_write.
Added :func:repr support to :class:bson.tz_util.FixedOffset.
Fixed a bug where PyMongo would raise InvalidBSON: unhashable type: 'tzfile'
when using :attr:~bson.codec_options.DatetimeConversion.DATETIME_CLAMP or
:attr:~bson.codec_options.DatetimeConversion.DATETIME_AUTO with a timezone from dateutil.
Fixed a bug where PyMongo would raise InvalidBSON: date value out of range
Deprecation of "namespace-" terminology, replaced by "keyspace-" (removal in 2.0)
deprecation of all namespace method names
deprecation of the namespace= named argument to all methods
deprecation of the update_db_namespace parameter to create_*space
Deprecation of collection bulk_write method (removal in 2.0)
APICommander logs warnings received from the Data API
Full removal of "core library" from the current API:
DevOps API accessed through APICommander everywhere
Admin objects use APICommander consistently
[Async]Database and [Async]Collection directly use APICommander
Cursor library uses APICommander directly
Core library imports triggers a submodule-wide deprecation warning
(simplification of the vector/vectorize deprecator utility)
Widened exception hierarchy with:
DevOpsAPIHttpException
DevOpsAPITimeoutException
DevOpsAPIFaultyResponseException
Rearrangement into separate modules for:
constants, strings, magic numbers and settings
request low-level tools
payload/response transformations
(sometimes with temporary duplication to avoid depending on 'core')
Testing:
testing on HCD targets Data API v 1.0.16
added tests for APICommander
improved tests for admin classes
Logging of API requests made more uniform and easier to read
Replaced collections.abc.Iterator => typing.Iterator for python3.8 compatibility
Deprecation of "namespace-" terminology, replaced by "keyspace-" (removal in 2.0)
- deprecation of all namespace method names
- deprecation of the namespace= named argument to all methods
- deprecation of the update_db_namespace parameter to create_*space
Deprecation of collection bulk_write method (removal in 2.0)
APICommander logs warnings received from the Data API
Full removal of "core library" from the current API:
- DevOps API accessed through APICommander everywhere
- Admin objects use APICommander consistently
- [Async]Database and [Async]Collection directly use APICommander
- Cursor library uses APICommander directly
- Core library imports triggers a submodule-wide deprecation warning
- (simplification of the vector/vectorize deprecator utility)
Widened exception hierarchy with:
- DevOpsAPIHttpException
- DevOpsAPITimeoutException
- DevOpsAPIFaultyResponseException
Rearrangement into separate modules for:
- constants, strings, magic numbers and settings
- request low-level tools
- payload/response transformations
- (sometimes with temporary duplication to avoid depending on 'core')
Testing:
- testing on HCD targets Data API v 1.0.16
- added tests for APICommander
- improved tests for admin classes
Logging of API requests made more uniform and easier to read
Replaced collections.abc.Iterator => typing.Iterator for python3.8 compatibility
Commits
bd99707 Deprecation of all things "namespace" (#316)
bdf08a4 Complete removal of "core" from the idiomatic API (#314)
[orm] [bug] [typing] Fixed issue where it was not possible to use typing.Literal with
Mapped[] on Python 3.8 and 3.9. Pull request courtesy Frazer McLean.
[orm] [bug] Fixed issue in ORM evaluator where two datatypes being evaluated with the
SQL concatenator operator would not be checked for
UnevaluatableError based on their datatype; this missed the case
of _postgresql.JSONB values being used in a concatenate operation
which is supported by PostgreSQL as well as how SQLAlchemy renders the SQL
for this operation, but does not work at the Python level. By implementing
UnevaluatableError for this combination, ORM update statements
will now fall back to "expire" when a concatenated JSON value used in a SET
clause is to be synchronized to a Python object.
[orm] [bug] An warning is emitted if _orm.joinedload() or
_orm.subqueryload() are used as a top level option against a
statement that is not a SELECT statement, such as with an
insert().returning(). There are no JOINs in INSERT statements nor is
there a "subquery" that can be repurposed for subquery eager loading, and
for UPDATE/DELETE joinedload does not support these either, so it is never
appropriate for this use to pass silently.
[orm] [bug] Fixed issue where using loader options such as _orm.selectinload()
with additional criteria in combination with ORM DML such as
_sql.insert() with RETURNING would not correctly set up internal
contexts required for caching to work correctly, leading to incorrect
results.
[mysql] [bug] Fixed issue in mariadbconnector dialect where query string arguments that
weren't checked integer or boolean arguments would be ignored, such as
string arguments like unix_socket, etc. As part of this change, the
argument parsing for particular elements such as client_flags,
compress, local_infile has been made more consistent across all
v0.30.3: CogVideoX Image-to-Video and Video-to-Video
This patch release adds Diffusers support for the upcoming CogVideoX-5B-I2V release (an Image-to-Video generation model)! The model weights will be available by end of the week on the HF Hub at THUDM/CogVideoX-5b-I2V (Link). Stay tuned for the release!
This release features two new pipelines:
CogVideoXImageToVideoPipeline
CogVideoXVideoToVideoPipeline
Additionally, we now have support for tiled encoding in the CogVideoX VAE. This can be enabled by calling the vae.enable_tiling() method, and it is used in the new Video-to-Video pipeline to encode sample videos to latents in a memory-efficient manner.
CogVideoXImageToVideoPipeline
The code below demonstrates how to use the new image-to-video pipeline:
import torch
from diffusers import CogVideoXImageToVideoPipeline
from diffusers.utils import export_to_video, load_image
prompt = "An astronaut hatching from an egg, on the surface of the moon, the darkness and depth of space realised in the background. High quality, ultrarealistic detail and breath-taking movie-like camera shot."
image = load_image(
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/astronaut.jpg"
)
video = pipe(image, prompt, use_dynamic_cfg=True)
export_to_video(video.frames[0], "output.mp4", fps=8)
CogVideoXVideoToVideoPipeline
The code below demonstrates how to use the new video-to-video pipeline:
import torch
from diffusers import CogVideoXDPMScheduler, CogVideoXVideoToVideoPipeline
from diffusers.utils import export_to_video, load_video
</tr></table>
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself)
- `@dependabot ignore minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself)
- `@dependabot ignore ` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore ` will remove all of the ignore conditions of the specified dependency
- `@dependabot unignore ` will remove the ignore condition of the specified dependency and ignore conditions
Bumps the dependencies group with 11 updates:
1.45.11.47.15.9.25.9.40.24.70.25.11.35.191.35.244.8.04.9.13.8.03.8.11.8.11.9.01.11.11.11.21.4.21.5.02.0.342.0.350.30.20.30.3Updates
openaifrom 1.45.1 to 1.47.1Release notes
Sourced from openai's releases.
Changelog
Sourced from openai's changelog.
Commits
89e5a01release: 1.47.15b2dbcdfix(pydantic v1): avoid warnings error45315a7release: 1.47.0eab2e5achore(types): improve type name for embedding models (#1730)3765cc2feat(client): send retry count header6172976release: 1.46.1bd1e9e2fix(client): handle domains with underscores (#1726)349b7f6chore(streaming): silence pydantic model_dump warnings (#1722)bcf9fccrelease: 1.46.04b30234feat(client): add ._request_id property to object responses (#1707)Updates
coherefrom 5.9.2 to 5.9.4Updates
huggingface-hubfrom 0.24.7 to 0.25.1Release notes
Sourced from huggingface-hub's releases.
... (truncated)
Commits
f7dc3fdRelease: v0.25.1efafba2Raise error if encountered in chat completion SSE stream (#2558)10e403dRelease: v0.25.0e331a05Release: v0.25.0.rc11b4bfe7Merge branch 'main' into v0.25-releasef1a7ed4Suggest using upload_large_folder when appropriate (#2547)796eb5aFixed the issue #2535 Added user followers and following in class User and ad...a06e246Reduce number of commits in upload large folder (#2546)6f49af5Release: v0.25.0.rc0a9adf8fAdd GGUF as supported expand[] parameter (#2545)Updates
boto3from 1.35.19 to 1.35.24Commits
0902ce1Merge branch 'release-1.35.24'03cb1edBumping version to 1.35.2408b0363Add changelog entries from botocore746a999Merge branch 'release-1.35.23'bf912c6Merge branch 'release-1.35.23' into developdc7b107Bumping version to 1.35.231edd472Add changelog entries from botocoredd54fe9Merge branch 'release-1.35.22'9acc473Merge branch 'release-1.35.22' into developfe83dceBumping version to 1.35.22Updates
pymongofrom 4.8.0 to 4.9.1Release notes
Sourced from pymongo's releases.
Changelog
Sourced from pymongo's changelog.
... (truncated)
Commits
8b26d4bBUMP 4.9.1d0772f2PYTHON-4773 - Async PyMongo Beta docs update (#1868)2ddd16dBUMP 4.10.0.dev0699d962BUMP 4.92c432b5PYTHON-4768 - Fix atlas connection tests and cleanup uses of raw MongoClients...6d472a1PYTHON-4738 Skip encryption test_fork on PyPy (#1865)9a71be1PYTHON-4740 Convert asyncio.TimeoutError to socket.timeout for compat (#1864)c136684PYTHON-4585 Cursor.to_list does not apply client's timeoutMS setting (#1860)40ebc16PYTHON-4764 Update to use current supported EVG hosts (#1858)163e3d4PYTHON-4738 - Make test_encryption.TestClientSimple.test_fork sync-only (#1862)Updates
marqofrom 3.8.0 to 3.8.1Commits
Updates
elevenlabsfrom 1.8.1 to 1.9.0Release notes
Sourced from elevenlabs's releases.
Commits
317b471(chore): remove pydantic aliases (#364)a9e6d06(fix): remove pydantic warningsUpdates
qdrant-clientfrom 1.11.1 to 1.11.2Release notes
Sourced from qdrant-client's releases.
Commits
294f675bump version to 1.11.208a3ec4Fix nested prefetches in qdrant_local.py (#772)4070e34fix: fix get vectors (#775)f9ece82Fix grpc payload index params (#773)169094bTsk 757 add support for tuple in payload (#763)Updates
astrapyfrom 1.4.2 to 1.5.0Release notes
Sourced from astrapy's releases.
Changelog
Sourced from astrapy's changelog.
Commits
bd99707Deprecation of all things "namespace" (#316)bdf08a4Complete removal of "core" from the idiomatic API (#314)Updates
sqlalchemyfrom 2.0.34 to 2.0.35Release notes
Sourced from sqlalchemy's releases.
... (truncated)
Commits
Updates
diffusersfrom 0.30.2 to 0.30.3Release notes
Sourced from diffusers's releases.
... (truncated)
Commits
c9ff360Release: v0.30.327b5786CogVideoX-5b-I2V support (#9418)c7407a3[refactor] move positional embeddings to patch embed layer for CogVideoX (#9263)7be2611[CI] Quick fix for Cog Video Test (#9373)c980e4b[core] CogVideoX memory optimizations in VAE encode (#9340)6dd94b3[core] Support VideoToVideo with CogVideoX (#9333)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show