Setting a different client api endpoiint for Bigquery

GoogleCloudDataproc / spark-bigquery-connector

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Apache License 2.0

374 stars 196 forks source link

Setting a different client api endpoiint for Bigquery #1296

Open EmilioIppoliti opened 1 week ago

EmilioIppoliti commented 1 week ago

Hi all ,

I created a provate endpoint for my Google Bigquery instance , and I wolud like to connect to this endpoint using the spark-bigquery-connector in Python . At the moment it seems there isn't any possibility to set a different endpoit on the Spark options uding bigquery formart , Could you me suggest something to resolve this issue ?

As illustrated in the previous image , I was able to setup only the proxy but no private endpoint .

Thanks

davidrabinowitz commented 1 week ago

You can set the following options:

bigQueryHttpEndpoint - the endpoint for the BigQuery V2 API REST endpoint.
bigQueryStorageGrpcEndpoint - The endpoint for the Spark BigQuery Storage API, used to read and write the data (DIRECT write)

Any reason you need to do that?