search

cloudera-labs / hms-mirror

"hms-mirror" is a utility used to bridge the gap between two clusters and migrate hive metadata.
Apache License 2.0
13 stars 8 forks source link

Add option to enabled / disable stats collection on sql transfers #59

Closed dstreev closed 1 year ago

dstreev commented 1 year ago

hive.stats.autogather and hive.stats.column.autogather should have options to set when doing SQL inserts.

If disabled, which should be the default, we should build a script to run the stats collection on the tables in another file.

dstreev commented 1 year ago

This is set in the configuration file for the cluster (LEFT and RIGHT). For the LEFT, set this to deal with transfer data movement. The settings are 'false' by default, which will turn off stats for the inserts.

clusters:
  LEFT|RIGHT:
    enableAutoTableStats: false|true
    enableAutoColumnStats: false|true
dstreev commented 1 year ago

See docs: https://github.com/cloudera-labs/hms-mirror/blob/main/README.md#auto-gathering-stats-disabled-by-default