A fairly sizeable rewrite of how the sincedb works, full details in the ARCHITECTURE.md
This resolves #74 where records were going missing in a high-volume situation
The change is that rather than have a single "this was the last timestamp for some log group" the change is to maintain a window of N minutes worth of events and use it to:
Pick a start time for 'filter_log_events' so we won't skip over records
Avoid reprocessing the same records twice
It replaces my last PR for this, as it still had some problems PR#92
A fairly sizeable rewrite of how the sincedb works, full details in the ARCHITECTURE.md
This resolves #74 where records were going missing in a high-volume situation
The change is that rather than have a single "this was the last timestamp for some log group" the change is to maintain a window of N minutes worth of events and use it to:
It replaces my last PR for this, as it still had some problems PR#92