search

apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.69k stars 4.2k forks source link

Remove dependency on google-apitools package in Python SDK. #19073

Open kennknowles opened 2 years ago

kennknowles commented 2 years ago

AFAIK google-apitools is not officially supported. For Google Cloud Platform APIs such as Datastore, Cloud Storage or Pub/Sub, using Cloud Client Libraries for Python is recommended. If that not possible, we may be able to use https://github.com/google/google-api-python-client - officially supported, but now in maintenance mode.

Imported from Jira BEAM-4850. Original Jira may contain additional context. Reported by: tvalentyn.

BjornPrime commented 1 year ago

.take-issue

BjornPrime commented 1 year ago

Will be using this as a tracking issue for removing the apitools-generated clients and other stray references to apitools, so that we can eventually remove it as a dependency.

BjornPrime commented 1 year ago

Replace storage_v1_client with GCS client #25676

BjornPrime commented 1 year ago

Replace bigquery_v2_client with Google Cloud BigQuery client (and associated libraries) #25678

BjornPrime commented 1 year ago

Replace dataflow_v1b3_client with a Google Cloud Python client #25679

BjornPrime commented 1 year ago

Replace cloudbuild_v1_client with Google Cloud Build client #25680

BjornPrime commented 1 year ago

Remove remaining dependency on apitools #25681

Abacn commented 1 year ago

Some context: https://s.apache.org/beam-python-dependencies-pm in particular this action item