Preload batch size is artificially reduced as we count in-progress rows with a pending AS action twice

[MathieuLamiot]

Context I identified the issue when rewriting integration tests for https://github.com/wp-media/wp-rocket/issues/6396 When deciding on the preload batch size, two limitations are implemented:

• We reduce the max. batch size by the number of actions in the AS queue in process_pending_jobs()
• We reduce the batch size by the number of in-progress actions in the DB in get_pending_jobs() As a result:
• If a page has a pending AS action, and is in-progress in the DB (which is an expected behavior), then it is counted twice as a reduction of the batch size.

Expected behavior The effective maximum batch size must be rocket_preload_cache_pending_jobs_cron_rows_count - number of pending actions in the AS queue - in_progress rows that don't have a pending actions in the AS queue.

Acceptance Criteria Test case InProgressShouldNotExceedMaxQueue must be re-written exactly as follows and passing:

• rocket_preload_cache_pending_jobs_cron_rows_count must be set to 4

[piotrbak]

@MathieuLamiot This is only about the tests, right?

[MathieuLamiot]

No it's actually impacting the way we do preload. It is still valid and will be even after #6396. The impact is that preload can be slower than expected as we count twice an in-progress preload in the batch size. So if you start a new batch (expected: 45 items) but there are already 6 in-progress, I think we only add 33 new items instead of 39.