Feature: New column name transformation - replace_period_with_underscore
BQ doesn't support period in column_names. Similar to quote, lower, add_underscore_when_invalid and snake_case, this PR introduces a new column transformation to replace period with underscore.
A tangible use case
A pipeline reading in data from Google Ads Query Language using a google ads tap. Since the result contains column name with period, we cannot directly load into BQ due to the naming restriction. We have to manually map column names with period to underscore under stream_maps (requiring for a type casting), which becomes unwieldy when we want to load multiple tables/columns
z3z1ma
commented
2 months ago
Feature: New column name transformation -
replace_period_with_underscoreBQ doesn't support
periodin column_names. Similar toquote,lower,add_underscore_when_invalidandsnake_case, this PR introduces a new column transformation to replace period with underscore.A tangible use case
A pipeline reading in data from Google Ads Query Language using a google ads tap. Since the result contains column name with
period, we cannot directly load into BQ due to the naming restriction. We have to manually map column names with period to underscore under stream_maps (requiring for a type casting), which becomes unwieldy when we want to load multiple tables/columnscc: @z3z1ma @blancaruano for some 👀