[Api?] Add Configuration option for shared ArrayPool limits

[benaadams]

Add Configuration option for shared ArrayPool limits

Currently the number of array buffers per size is MaxBuffersPerArraySizePerCore * ProcessorCount

With const MaxBuffersPerArraySizePerCore = 8

Change to:

readonly static MaxBuffersPerArraySizePerCore = 
                   RoundedUpPowerOf2ConfigValue ?? DefaultBuffersPerArraySizePerCore;

Rationale

With more and more activities using the Shared ArrayPool there is a higher chance of it becoming depleted; allocating extra, then discarding the over allocation on return.

The "over" allocation will act as slightly slower to allocate regular buffers; but offer no advantage over regular non-pooled memory.

The characteristics of the buffer usage for an application may be known via observation; so for example if you know you will have on average (or short term 95 percentile burst cycles) of 500+ on going buffer ops by framework functions (async reads etc); then you could up the per core config value to cater for this.

Or if you are running a single app on a machine with a plethora of RAM you might just do it for fun.

[karelz]

cc @stephentoub @jkotas @brianrob

[jkotas]

The same concern about hints on how much memory to keep warm applies to GC in general, not just ArrayPool.

I do not think that the current implementation of ArrayPool is the final one. I would much rather look at how to make the ArrayPool self-tuning, or cooperate with the GC on self-tuning. The problem with similar knobs is that very few people knows how to set them well; or they set them once and then forget about it and it starts to actively hurt them as the workload changes. ArrayPool is basically just C# flavor of malloc/free. We should be able to do as well, or even better than C# malloc/free does. As far as I know, C++ malloc/free does not require similar tuning knobs to perform well, so we should not need one either.

I can see why these knobs may be useful for the current implementation of ArrayPool in some cases. If the tuning knob makes a big measurable difference short term, I would be ok with adding it as environment variable. Not as a API that we will need to keep around forever.

[benaadams]

Yeah I was thinking as environment var; don't see it as an at runtime setting. And if it can tune using heuristics at a later date; happy both the be part of the real world testing and to have the enviorment var taken away.

GC is a more difficult beast as its a more general purpose allocator and cleaner; whereas the array pool is "easier" as it is focused on one very specific task of array allocation. Though I suppose there could become an array focused allocator in the GC (and might be already) which could take the place of the array pool longer term.

[alexperovich]

Using an environment variable makes sense to me, I don't want an API that will need to exist for the rest of time for compatibility. I don't have a good name to use though.

[ViktorHofer]

Is this still planned? It would be great if we could fine tune the shared ArrayPool via an environment variable, especially the max array size which is currently limited to 2MB.