feat: Batch processing for an anchor loop

[ukstv]

batch size = concurrency

[linear[bot]]

WS2-2974 Add concurrency to anchor polling loop

[stbrody]

This works, though I have two (admittedly both on the relatively minor side) issues with the approach.

One concern is that combining the batch size used for fetching requests from the store and the concurrency limit of processing those requests into a single knob limits us. I think it makes sense to fetch a large number of requests from the store at once (like the 1000 value we previously had here), but we probably want to limit the concurrency of how many requests get processed at once quite a bit more than that, since we don't want to overwhelm the CAS or use too much memory on the Ceramic node holding all the CAR file responses from that many requests in memory at once.

The other possible improvement I see over this approach is that rather than doing things in batches where every single request from the batch must be fully processed before we start any requests in a new batch, we can instead specify the number of requests we want processing concurrently and then keep a steady supply of requests always in processing up to that limit. That can keep us busier, processing things faster, and smoothing out the load.

I took a crack at implementing an alternative approach that addresses these two issues here: https://github.com/ceramicnetwork/js-ceramic/pull/3153, PTAL.

[ukstv]

Okay.