hashicorp-forge / grove

A Software as a Service (SaaS) log collection framework.
https://hashicorp-forge.github.io/grove/
Mozilla Public License 2.0
130 stars 11 forks source link

🐛 Fix pagination bug in Tines connector (related to #52). #60

Closed hcpadkins closed 3 weeks ago

hcpadkins commented 3 weeks ago

Overview

This pull-request resolves a bug reported by @britton-from-notion in #52. This has been validated to resolve the reported issue in a local reproduction environment.

Basically, it's a silly bug where the page number is being incorrectly set to the value of next_page rather than next_page_number - where one is a URI, and the other is an integer.

As a result, we're passing a URI to the page directive of the Tines API, rather than just an integer. Looking at my old test fixtures, it appears that this next_page_number field is "new", but it appears that the next_page field was always a URI so it was still an error in the connector. We just never triggered it as the event load when using the connector was always less than 500-events per run interval.

Unfortunately, it seems like the Tines API is silently accepting a URL in the page field, but appears to be ignoring the value entirely and just returning the first page ad-infinitum.