search

mozilla / gcp-ingestion

Documentation and implementation of telemetry ingestion on Google Cloud Platform
https://mozilla.github.io/gcp-ingestion/
Mozilla Public License 2.0
75 stars 31 forks source link

Bump org.apache.beam:beam-sdks-java-google-cloud-platform-bom from 2.51.0 to 2.53.0 #2534

Closed dependabot[bot] closed 6 months ago

dependabot[bot] commented 8 months ago

Bumps org.apache.beam:beam-sdks-java-google-cloud-platform-bom from 2.51.0 to 2.53.0.

Release notes

Sourced from org.apache.beam:beam-sdks-java-google-cloud-platform-bom's releases.

Beam 2.53.0 release

We are happy to present the new 2.53.0 release of Beam. This release includes both improvements and new functionality. See the download page for this release.

For more information on changes in 2.53.0, check out the detailed release notes.

Highlights

  • Python streaming users that use 2.47.0 and newer versions of Beam should update to version 2.53.0, which fixes a known issue: (#27330).

I/Os

  • TextIO now supports skipping multiple header lines (Java) (#17990).
  • Python GCSIO is now implemented with GCP GCS Client instead of apitools (#25676)
  • Adding support for LowCardinality DataType in ClickHouse (Java) (#29533).
  • Added support for handling bad records to KafkaIO (Java) (#29546)
  • Add support for generating text embeddings in MLTransform for Vertex AI and Hugging Face Hub models.(#29564)
  • NATS IO connector added (Go) (#29000).

New Features / Improvements

  • The Python SDK now type checks collections.abc.Collections types properly. Some type hints that were erroneously allowed by the SDK may now fail. (#29272)
  • Running multi-language pipelines locally no longer requires Docker. Instead, the same (generally auto-started) subprocess used to perform the expansion can also be used as the cross-language worker.
  • Framework for adding Error Handlers to composite transforms added in Java (#29164).
  • Python 3.11 images now include google-cloud-profiler (#29561).

Breaking Changes

Deprecations

  • Euphoria DSL is deprecated and will be removed in a future release (not before 2.56.0) (#29451)

Bugfixes

  • (Python) Fixed sporadic crashes in streaming pipelines that affected some users of 2.47.0 and newer SDKs (#27330).
  • (Python) Fixed a bug that caused MLTransform to drop identical elements in the output PCollection (#29600).

List of Contributors

According to git shortlog, the following people contributed to the 2.53.0 release. Thank you to all contributors!

Ahmed Abualsaud

Ahmet Altay

... (truncated)

Changelog

Sourced from org.apache.beam:beam-sdks-java-google-cloud-platform-bom's changelog.

[2.53.0] - 2024-01-04

Highlights

  • Python streaming users that use 2.47.0 and newer versions of Beam should update to version 2.53.0, which fixes a known issue: (#27330).

I/Os

  • TextIO now supports skipping multiple header lines (Java) (#17990).
  • Python GCSIO is now implemented with GCP GCS Client instead of apitools (#25676)
  • Added support for handling bad records to KafkaIO (Java) (#29546)
  • Add support for generating text embeddings in MLTransform for Vertex AI and Hugging Face Hub models.(#29564)
  • NATS IO connector added (Go) (#29000).
  • Adding support for LowCardinality (Java) (#29533).

New Features / Improvements

  • The Python SDK now type checks collections.abc.Collections types properly. Some type hints that were erroneously allowed by the SDK may now fail. (#29272)
  • Running multi-language pipelines locally no longer requires Docker. Instead, the same (generally auto-started) subprocess used to perform the expansion can also be used as the cross-language worker.
  • Framework for adding Error Handlers to composite transforms added in Java (#29164).
  • Python 3.11 images now include google-cloud-profiler (#29561).

Breaking Changes

Deprecations

  • Euphoria DSL is deprecated and will be removed in a future release (not before 2.56.0) (#29451)

Bugfixes

  • (Python) Fixed sporadic crashes in streaming pipelines that affected some users of 2.47.0 and newer SDKs (#27330).
  • (Python) Fixed a bug that caused MLTransform to drop identical elements in the output PCollection (#29600).

[2.52.0] - 2023-11-17

Highlights

  • Previously deprecated Avro-dependent code (Beam Release 2.46.0) has been finally removed from Java SDK "core" package. Please, use beam-sdks-java-extensions-avro instead. This will allow to easily update Avro version in user code without potential breaking changes in Beam "core" since the Beam Avro extension already supports the latest Avro versions and should handle this. (#25252).
  • Publishing Java 21 SDK container images now supported as part of Apache Beam release process. (#28120)
    • Direct Runner and Dataflow Runner support running pipelines on Java21 (experimental until tests fully setup). For other runners (Flink, Spark, Samza, etc) support status depend on runner projects.

New Features / Improvements

... (truncated)

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
dependabot[bot] commented 6 months ago

Superseded by #2549.