PeerDB-io / peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
https://peerdb.io
Other
2.17k stars 88 forks source link

Replace 32 sync limit with ContinueAsNewSuggested #2014

Open serprex opened 1 month ago

serprex commented 1 month ago

Experiment. Still unsure since long history impacts ui latency, but nice to have something that'll tune restart frequency based on workload

heavycrystal commented 6 days ago

another thing: SyncFlow also restarts independently (set by maxSyncsPerSyncFlow) should handle that so this works properly