After analysis, we found that the ideal batch size varies for each table this PR adds the ability to have DataIterationBatchSizePerTableOverride to be configurable per table.
We can pass in map[schema_name]: map[table_name]: batchSize to the config to set the batchSize for each table. If no batchSize is specified, default 200 is used.
After analysis, we found that the ideal batch size varies for each table this PR adds the ability to have
DataIterationBatchSizePerTableOverrideto be configurable per table.We can pass in
map[schema_name]: map[table_name]: batchSizeto the config to set the batchSize for each table. If no batchSize is specified, default200is used.