Emitting state more often as soon as something got loaded into snowflake and copied other missing functionalities from target-snowflake:
flush_all_streams: (Default: False) Flush and load every stream into Snowflake when one batch is full. Warning: This may trigger the COPY command to use files with low number of records, and may cause performance problems.
parallelism: (Default: 0) The number of threads used to flush tables. 0 will create a thread for each stream, up to parallelism_max. -1 will create a thread for each CPU core. Any other positive number will create that number of threads, up to parallelism_max.
parallelism_max: (Default: 16) Max number of parallel threads to use when flushing tables.
Emitting state more often as soon as something got loaded into snowflake and copied other missing functionalities from target-snowflake:
parallelism_max.