search

auth0 / docs

Auth0 documentation
MIT License
372 stars 1.62k forks source link

PaperTrail extension documentation - batch size explanation #7013

Open ryno1234 opened 5 years ago

ryno1234 commented 5 years ago

The PaperTrail documentation explains each of the fields necessary to configure the extension, but glosses over a detail I think is vitally important to understand.

The BATCH_SIZE needs further explanation. Currently, the docs explain that BATCH_SIZE is the number of log messages that will be transferred per each execution of the scheduled job. I have mine set at 100, the max.

I have my schedule set to every 5 minutes.

Let's say I have 110 log messages that occurred in 5 minute period between scheduled job execution 1 and execution 2. Does the next execution (3) pick up where it left off and grab those 10 extra records that weren't sent on the previous execution?

Further, what happens if we are regularly producing more than 100 log messages within the 5 minute time frame. This seems to point to the idea that we would never "catch up" to real-time and consistently be shipping logs that are further and further out of date.

khorne3 commented 5 years ago

Hi @ryno1234! Thanks for letting us know. I've added this issue to our backlog, and we will be updating this in the upcoming weeks.