GoogleCloudPlatform / k8s-config-connector

GCP Config Connector, a Kubernetes add-on for managing GCP resources
https://cloud.google.com/config-connector/docs/overview
Apache License 2.0
901 stars 234 forks source link

Enable "Use Table Schema" for Pub/Sub subscriptions #1912

Open sthomp12 opened 5 months ago

sthomp12 commented 5 months ago

Checklist

Describe the feature or resource

For Pub/Sub BigQuery Subscriptions we would like to request the enable the use of the "Use Table Schema" flag in config-connector. This flag is described here and has been enabled in terraform for Pub/Sub BigQuery Subscriptions.

When this flag is set to true the subscription will use the BigQuery table's schema as the columns when writing to BigQuery.

Additional information

No response

Importance

We have found an issue when using BigQuery Subscriptions where a field has been set to data type float. There was a loss in precision of the float value when streaming from the subscription to the table. However, we noticed when using this option "use table schema" this loss of precision did not happen.

Our resources, such as BigQuery Subscriptions, are created via config-connector and we cannot currently set this value to true.

SahandJ commented 2 weeks ago

Hi. I've encountered the same issue, we would like to use the table schema when creating a pubsub bigquery-subscription. Is there any updates for this?

karolinajoksaite commented 1 week ago

@SahandJ no update yet but I will update this thread if/when I get one!