Previously, the library was unable to switch to the new generation. The reason was that when a generation change happens, it stops the Worker reading the previous generation. While stopping the Worker, it shuts down the Executor. Afterwards, a new Worker (for new generation) is started, but it reuses the previous Executor. Once you shut down the Executor, you can't restart it or schedule new TaskActions on it.
The solution is to re-create the Executor every time. The side effect of my change is that previously the user could provide their own ExecutorService and this is now removed. It might be added back in the future, once we think of a good solution to how the user could provide their own ExecutorService (this is more an API design problem; this does not seem like a high-priority thing for now).
There are no integration tests added in this PR. I fully agree that there should be added. However, it would require substantial amount of new "infrastructure/framework" to test: starting test multi-node clusters, testing high-level scylla-cdc-lib (previously I was testing only those lower-level driver methods). I hope I will be able to push a PR with those tests in the near future.
Fix generation change in scylla-cdc-lib.
Previously, the library was unable to switch to the new generation. The reason was that when a generation change happens, it stops the Worker reading the previous generation. While stopping the Worker, it shuts down the Executor. Afterwards, a new Worker (for new generation) is started, but it reuses the previous Executor. Once you shut down the Executor, you can't restart it or schedule new TaskActions on it.
The solution is to re-create the Executor every time. The side effect of my change is that previously the user could provide their own ExecutorService and this is now removed. It might be added back in the future, once we think of a good solution to how the user could provide their own ExecutorService (this is more an API design problem; this does not seem like a high-priority thing for now).
There are no integration tests added in this PR. I fully agree that there should be added. However, it would require substantial amount of new "infrastructure/framework" to test: starting test multi-node clusters, testing high-level scylla-cdc-lib (previously I was testing only those lower-level driver methods). I hope I will be able to push a PR with those tests in the near future.