It seems that there might be a possibility of an infinite loop when syncing records. This was discovered when investigating
a large number of records syncing on the events endpoint:
Though I'm not an engineer myself, if the max_modified (eventAt) date time remains the same within the set of 10,000 records over a requested page of events; then the offset would be reset on the same page.
The extraction would then repeat up to an offset of 10000; and since the max_modified date-time hadn't changed, the offset would drop to 0 again.
It seems that there might be a possibility of an infinite loop when syncing records. This was discovered when investigating a large number of records syncing on the
eventsendpoint:https://github.com/singer-io/tap-outreach/blob/60091fac46ee6401c29ba9a0767aaa17861b857f/tap_outreach/sync.py#L270-L272
Though I'm not an engineer myself, if the max_modified (eventAt) date time remains the same within the set of 10,000 records over a requested page of events; then the offset would be reset on the same page.
The extraction would then repeat up to an offset of 10000; and since the max_modified date-time hadn't changed, the offset would drop to 0 again.
This would continue, infinitely.