If you observe the url, the paginated page ends with some string like P23, P24, ...etc, this seems to be a reliable source showing which page we are at; it can also be used to "jump" to that page.
We may record the next page for next session (where current session is about to reach its limitation like 3K). But how to wait for awhile and let the next session start is a question. But perhaps we don't need to worry about this because it can be queued. But we may have increasing concerns about checking whether a scraper is working or not, since its "company-wise" task is taking longer and longer, in this sense.
So our scraper can scale.
Some useful ideas
P23
,P24
, ...etc, this seems to be a reliable source showing which page we are at; it can also be used to "jump" to that page.