Safer pagination and refresh for site cache

[timotheeg]

Problem

The way isomercms retrieves and cache the list of repos is with the help of an environment variable ISOMERPAGES_REPO_PAGE_COUNT. That variable contains a number, which is used to create parallel requests to github.

In practice that works, but operationally this is not great:

• It introduces a mental load to update the variables if the number of supported repo grows beyond the current page count (easy to forget and create a potential incident!)
• The concurrency fetches optimize for time in all cases, while for the internal cache refreshes, optimizing for load distribution would be safer

Github APIs have a very robust pagination system, so we can derive the number of pages from that to make the system dynamic, and remove the (small) operational risk of forgetting to update the env var.

Additionally, the cache only has one exposed API, which is to retrieve the lastUpdated time by repoName. The list of repo is not stored in cache by name, but it is a list and so the lookup does a list search every time.

Solution

1. split the getAllRepoData() calls into 2 distinct use cases: a. exported API optimized for time: we fetch the first page, assess how many pages there in total, and fetch all remaining pages in parallel b. optimized for load distribution: we fetch the pages sequentially by following github pagination system's next link
2. populate the cache at bootstrap time with the time-optimized call
3. cache refresh uses the load-optimized call
4. Use a dictionary to store the repos indexes by name for O(1) lookup

Notes:

1. The time-optimized fetch is still slower than the previous version, since the first page call is always made sequentially
2. We could (to discuss) cache the number of pages, such that: a. only the first bootstrap call has the 2 step fetch b. every refresh call updates the number of pages c. when the number of pages is cached, the fast getAllRepoData() does all its calls in parallel again.
3. We add a custom span to instrument the time the cache refresh process takes.

Breaking Changes

• [ ] Yes - this PR contains breaking changes
• [X] No - this PR is backwards compatible with ALL of the following feature flags in this doc

Reference:

• slack thread where this was discussed

Results: Sample trace for the new refresh process that shows fetches the pages sequentially works:

Post Deploy Work (after stability is observed)

• [ ] delete the SSM entries 'PROD_ISOMERPAGES_REPO_PAGE_COUNT' and STAGING_ISOMERPAGES_REPO_PAGE_COUNT. This should ONLY be done when we are sure we will not need to roll back possibly wait one full release.

[mergify[bot]]

This pull request has been stale for more than 30 days! Could someone please take a look at it @isomerpages/iso-engineers

[timotheeg]

Note: Typically another way to deal with pages and fetching data is by using the github sdk octokit.js, which has pagination handling built in. I did not introduce it in this PR because the goal was just to remove the env var ISOMERPAGES_REPO_PAGE_COUNT, and I kept the parallel fetch for pages 2-N, which octokit wouldn't do.

That should probably be revisited at some point.

[timotheeg]

All comments addressed @seaerchin , Let me know if good to go. Thanks! 🙏

[timotheeg]

Result in prod:

• As expected: /v2/sites is slower now since the first page is always sequential (sample trace here)
• The refresh is a cheaper process now, since all the ages are sequential (sample internal trace)