We are happy to present the new 2.58.0 release of Beam.
This release includes both improvements and new functionality.
See the download page for this release.
Support for Solace source (SolaceIO.Read) added (Java) (#31440).
New Features / Improvements
Multiple RunInference instances can now share the same model instance by setting the model_identifier parameter (Python) (#31665).
Added options to control the number of Storage API multiplexing connections (#31721)
[BigQueryIO] Better handling for batch Storage Write API when it hits AppendRows throughput quota (#31837)
[IcebergIO] All specified catalog properties are passed through to the connector (#31726)
Removed a third-party LGPL dependency from the Go SDK (#31765).
Support for MapState and SetState when using Dataflow Runner v1 with Streaming Engine (Java) ([#18200])
Breaking Changes
[IcebergIO] IcebergCatalogConfig was changed to support specifying catalog properties in a key-store fashion (#31726)
[SpannerIO] Added validation that query and table cannot be specified at the same time for SpannerIO.read(). Previously withQuery overrides withTable, if set (#24956).
Bug fixes
[BigQueryIO] Fixed a bug in batch Storage Write API that frequently exhausted concurrent connections quota (#31710)
List of Contributors
According to git shortlog, the following people contributed to the 2.58.0 release. Thank you to all contributors!
Support for Solace source (SolaceIO.Read) added (Java) (#31440).
New Features / Improvements
Multiple RunInference instances can now share the same model instance by setting the model_identifier parameter (Python) (#31665).
Added options to control the number of Storage API multiplexing connections (#31721)
[BigQueryIO] Better handling for batch Storage Write API when it hits AppendRows throughput quota (#31837)
[IcebergIO] All specified catalog properties are passed through to the connector (#31726)
Removed a 3rd party LGPL dependency from the Go SDK (#31765).
Support for MapState and SetState when using Dataflow Runner v1 with Streaming Engine (Java) ([#18200])
Breaking Changes
[IcebergIO] IcebergCatalogConfig was changed to support specifying catalog properties in a key-store fashion (#31726)
[SpannerIO] Added validation that query and table cannot be specified at the same time for SpannerIO.read(). Previously withQuery overrides withTable, if set (#24956).
Bugfixes
[BigQueryIO] Fixed a bug in batch Storage Write API that frequently exhausted concurrent connections quota (#31710)
Fixed a logging issue where Python worker dependency installation logs sometimes were not emitted in a timely manner (#31977)
Ensure that BigtableIO closes the reader streams (#31477).
New Features / Improvements
Added Feast feature store handler for enrichment transform (Python) (#30957).
BigQuery per-worker metrics are reported by default for Streaming Dataflow Jobs (Java) (#31015)
Adds inMemory() variant of Java List and Map side inputs for more efficient lookups when the entire side input fits into memory.
Beam YAML now supports the jinja templating syntax.
Template variables can be passed with the (json-formatted) --jinja_variables flag.
DataFrame API now supports pandas 2.1.x and adds 12 more string functions for Series.(#31185).
Added BigQuery handler for enrichment transform (Python) (#31295)
Disable soft delete policy when creating the default bucket for a project (Java) (#31324).
Added DoFn.SetupContextParam and DoFn.BundleContextParam which can be used
as a python DoFn.process, Map, or FlatMap parameter to invoke a context
manager per DoFn setup or bundle (analogous to using setup/teardown
or start_bundle/finish_bundle respectively.)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
dependabot[bot]
commented
4 weeks ago
Bumps org.apache.beam:beam-sdks-java-google-cloud-platform-bom from 2.51.0 to 2.58.0.
Release notes
Sourced from org.apache.beam:beam-sdks-java-google-cloud-platform-bom's releases.
... (truncated)
Changelog
Sourced from org.apache.beam:beam-sdks-java-google-cloud-platform-bom's changelog.
... (truncated)
Commits
d8315f6Set version for 2.58.0 RC27fbbe0bAdds ordered list user state support to fnapi accessor cache. (#31923) (#32013)bb828dcRevert "[Cherrypick] Update Build Wheels to only build once on RCs (#32009)" ...0a4bdc3Update Build Wheels to only build once on RCs (#32009) (#32010)cdc19d0[yaml] Fix yaml provider schema validation and merging (#31974) (#31984)3ec5643Dedup Lineage and getTableToExtract call in BigQuerySourceBase (#31960) (#31972)6e19e38Use buffered loggers that periodically flush. (#31978)eca36f6Switch to use self-hosted runner for build_wheels action (#31866) (#31916)7626ad8Fix generateYamlDocs gradle task (#31909)92de083Export string sets in monitoring infos. (#31838) (#31849)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show