This pull request includes a minor bug fix to correctly create staging dataset.
In com.spotify.spark.bigquery.BigQueryClient, currently STAGING_DATASET_ID config includes STAGING_DATASET_LOCATION string by default, but if the location includes hyphen(e.g. asia-northeast1), then the dataset-id can't satisfy its naming rule1 and dataset creation fails.
So could you please add the conversion of hyphen into underscore before setting STAGING_DATASET_ID?
Hi,
This pull request includes a minor bug fix to correctly create staging dataset.
In
com.spotify.spark.bigquery.BigQueryClient, currentlySTAGING_DATASET_IDconfig includesSTAGING_DATASET_LOCATIONstring by default, but if the location includes hyphen(e.g. asia-northeast1), then the dataset-id can't satisfy its naming rule1 and dataset creation fails.So could you please add the conversion of hyphen into underscore before setting
STAGING_DATASET_ID?Thanks