Open xiangshiyin opened 1 month ago
With some further digging, we believe the key_properties
here should be the primary keys defined in the stream. With the current clustering key configuration in the sink, the BQ table clustering is directly influenced by the order of columns in the primary key combo defined in the incoming stream. It'll be good if we could have a configurable parameter under the sink so we have more flexibility.
cc. @epapineau
The doc about the config
cluster_on_key_properties
saysThe code confirms that
key_properties
is used to define clustering key set and used as the primary key in merge operationAnyone knows how to determine the default
key_properties
value and how to customize (is it even possible)? Thanks!