Setting a different client api endpoiint for Bigquery

GoogleCloudDataproc / spark-bigquery-connector

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Apache License 2.0

378 stars 198 forks source link

Setting a different client api endpoiint for Bigquery #1296

Closed EmilioIppoliti closed 1 month ago

EmilioIppoliti commented 2 months ago

Hi all ,

I created a provate endpoint for my Google Bigquery instance , and I wolud like to connect to this endpoint using the spark-bigquery-connector in Python . At the moment it seems there isn't any possibility to set a different endpoit on the Spark options uding bigquery formart , Could you me suggest something to resolve this issue ?

As illustrated in the previous image , I was able to setup only the proxy but no private endpoint .

Thanks

davidrabinowitz commented 2 months ago

You can set the following options:

bigQueryHttpEndpoint - the endpoint for the BigQuery V2 API REST endpoint.
bigQueryStorageGrpcEndpoint - The endpoint for the Spark BigQuery Storage API, used to read and write the data (DIRECT write)

Any reason you need to do that?