spotify / scio

A Scala API for Apache Beam and Google Cloud Dataflow.
https://spotify.github.io/scio
Apache License 2.0
2.56k stars 513 forks source link

Refactor bigtable API to use v2 client #5444

Open RustedBones opened 2 months ago

RustedBones commented 2 months ago

Update all bigtable APIs to use cloud v2 client.

Drop BigtableBulkWriter. This is TBD, but it looks to me can manually shard/groupBy and flatten to normal IO if they require this feature.

Change bigtable ScioContext api to use Admin.Table and Admin.Instance objects that cache clients instances.

codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 18.94737% with 77 lines in your changes missing coverage. Please review.

Project coverage is 61.59%. Comparing base (25c6987) to head (fc6db20). Report is 11 commits behind head on main.

Files Patch % Lines
...c/main/scala/com/spotify/scio/bigtable/Admin.scala 0.00% 55 Missing :warning:
...n/scala/com/spotify/scio/bigtable/BigTableIO.scala 50.00% 14 Missing :warning:
...otify/scio/bigtable/syntax/ScioContextSyntax.scala 20.00% 8 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #5444 +/- ## ========================================== + Coverage 61.29% 61.59% +0.29% ========================================== Files 311 312 +1 Lines 11062 11031 -31 Branches 774 780 +6 ========================================== + Hits 6780 6794 +14 + Misses 4282 4237 -45 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.