Track dirty pages in regionAllocator and preferentially reuse them when possible.

[dsm9000]

This is part 1 of 3, and is continued in:

• Part 2 of 3: https://github.com/snazzy-d/sdc/pull/336
• Part 3 of 3: https://github.com/snazzy-d/sdc/pull/337

Theory of operation: A segment of memory that has been released by the regionAllocator is considered dirty in its entirety. Memory newly obtained in refillAddressSpace is considered clean. Every Region now contains a dirtySize which corresponds to the dirty segment (if it exists) of that region. dirtySize is always less than or equal to size. Regions that are partially dirty (i.e. where 0 < dirtySize < size) may be created by the region merging logic in registerRegion.

A Region's dirtySize is mandatorily HugePageSize-aligned, precisely like size.

A Region containing dirt is guaranteed to contain precisely one contiguous run of dirt, residing at the bottom of the region.

Given as currently purging of dirty pages is not supported (it is introduced in https://github.com/snazzy-d/sdc/pull/337 ), the invariant holds that all dirt within a region is to be found strictly at the bottom of the region. Therefore, whenever a dirty region is reused by acquire, the respective dirtySize of both the utilized segment of memory and remainder segment (when one comes to exist) is easily calculated. Ergo all region dirt is tracked precisely at all times: only pages which are actually dirty are counted as dirty, and conversely. The allocator is at all times aware of all existing runs of dirty pages.

The total number of dirty pages existing at any given time may be obtained via regionAllocator.dirtyPageCount, which is accurate at all times, and always equal to the sum of the Region.dirtyPageCount from all usable regions known to the regionAllocator.

regionsByDirt at all times indexes every allocatable region that contains any dirt, ordered by its count of dirty pages, in descending order. regionsByClass now also returns (when queried via best-fit) the dirtiest region (i.e. having the largest number of dirty pages) that is simultaneously a best-fit by size class.

acquire now attempts to allocate from the overall longest dirty run known to the regionAllocator, when this is possible; and at all other times, allocates from the longest dirty run that is a size-class best-fit. Ergo, the maximum possible reuse of dirty pages by the regionAllocator is achieved.

[deadalnix]

Let's track in one PR and reuse in another. This has clearly gone out of hands. It seems to me like there is a lot of accidental complexity in there, but there is really no point arguing this when splitting thing in digestible chunk will make it obvious is there is or not.

[dsm9000]

@deadalnix https://github.com/snazzy-d/sdc/pull/338