We are happy to present the new 2.54.0 release of Beam.
This release includes both improvements and new functionality.
See the download page for this release.
Beam Java Batch pipelines run on Google Cloud Dataflow will default to the Portable (Runner V2)[https://cloud.google.com/dataflow/docs/runner-v2] starting with this version. (All other languages are already on Runner V2.)
This change is still rolling out to the Dataflow service, see (Runner V2 documentation)[https://cloud.google.com/dataflow/docs/runner-v2] for how to enable or disable it intentionally.
I/Os
Added support for writing to BigQuery dynamic destinations with Python's Storage Write API (#30045)
Adding support for Tuples DataType in ClickHouse (Java) (#29715).
Added support for handling bad records to FileIO, TextIO, AvroIO (#29670).
Added support for handling bad records to BigtableIO (#29885).
Beam Java Batch pipelines run on Google Cloud Dataflow will default to the Portable (Runner V2)[https://cloud.google.com/dataflow/docs/runner-v2] starting with this version. (All other languages are already on Runner V2.)
This change is still rolling out to the Dataflow service, see (Runner V2 documentation)[https://cloud.google.com/dataflow/docs/runner-v2] for how to enable or disable it intentionally.
I/Os
Added support for writing to BigQuery dynamic destinations with Python's Storage Write API (#30045)
Adding support for Tuples DataType in ClickHouse (Java) (#29715).
Added support for handling bad records to FileIO, TextIO, AvroIO (#29670).
Added support for handling bad records to BigtableIO (#29885).
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Bumps apache-beam[gcp] from 2.52.0 to 2.54.0.
Release notes
Sourced from apache-beam[gcp]'s releases.
... (truncated)
Changelog
Sourced from apache-beam[gcp]'s changelog.
... (truncated)
Commits
f660f49
Set version for 2.54.0 RC28978c80
Support dict as the input of delete_batch (#30241)435eab3
Add Default method for OutputReceiver.outputWindowedValue (#30240)782a78b
[CherryPick #30229] [Dataflow Streaming] Invalidate caches and remove work on...c726525
Fix JdbcIO PreCommit mysql breaking change (#30221)7cd144d
Marks failed bit as volatile since multiple threads read and write it. (#30216)f4f219a
[Cherry pick #30156 into 2.54.0] Make ReadFromBigQueryRequest id more randomi...7517e0e
Remove constraint introduced in #30059 (#30186) (#30189)1c974aa
[Cherry Pick #29979] Fix race condition in Dataflow sampler. (#30144)d1d3457
Evict closed readers from the cache. (#30133)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase
.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show