To allow to tune/customise the behaviour of one's source connector setup, I'd like to also have a config option poll.interval.ms in addition to scylla.query.time.window.size which defines effectively the query time window size + query interval for a 'live' / caught up worker task.
As per my understanding / reasoning the poll.interval.ms would/should be smaller than scylla.query.time.window.size - with the latter being applied while catching up / init phase.
Workers (connect tasks) ideally will evenly scatter queries for to the assigned array of streamIds / streamIdGroups (scylla-cdc-java worker task?).
Config Field Definition
poll.interval.ms
Positive integer value that specifies the frequency in milliseconds the connector should wait to poll for new data in each worker task (Vnode). Defaults to 15.000 milliseconds.
Type: Integer
Importance: Low
Default: 15000
Frequency in ms to poll for new data in each table.
Description
To allow to tune/customise the behaviour of one's source connector setup, I'd like to also have a config option
poll.interval.msin addition toscylla.query.time.window.sizewhich defines effectively the query time window size + query interval for a 'live' / caught up worker task.As per my understanding / reasoning the
poll.interval.mswould/should be smaller thanscylla.query.time.window.size- with the latter being applied while catching up / init phase.Workers (connect tasks) ideally will evenly scatter queries for to the assigned array of streamIds / streamIdGroups (scylla-cdc-java worker task?).
Config Field Definition
poll.interval.msPositive integer value that specifies the frequency in milliseconds the connector should wait to poll for new data in each worker task (Vnode). Defaults to 15.000 milliseconds.References