Improve evictions/retrieval policy

[Zannick]

The heap eventually gets to a point where it doesn't seem to make much progress; my last run got to 227M+57M rounds and the heap was still around 710000ms (total expected time) at the bottom, 706000ms in the db, progress in range 1-5 with most elements under 400000ms elapsed time, and that was fairly typical of it for many millions of rounds (with the upper limit being 1091000ms from the provided route).

We should be better about choosing which elements get removed from the heap when we overfill, since it seems like we have a lot of reshuffles. Right now, when we add a set of new items to the heap, we evict the first max_evictions elements above the min, starting from the bottommost progress level. This tends to cut out the middle and probably is also responsible for getting the small times evicted such that we need to do another retrieval right away.

We need to find some compromise: evicting only the highest elements of the heap overall will dump the ones with more progress that we're interested in attempting to find non-optimal solutions more quickly. We also can't merely evict the highest amount of each segment that we're about to add... simply because we have to evict more.

Here are some options:

• Evict in a triangular form: $n$ from progress 0, $n-1$ from progress 1, ... $1$ from progress n, until we've reached our chosen amount.
• Evict in a quadratic form: $n^2$ from progress 0, $(n-1)^2$ from progress 1, etc.
• Evict 1:1 from the respective queues that we're extending, then do one of the other strategies to evict more.
• Evict 1:1 from the respective queues until we surpass the min_evictions.
• Add all the elements to the heap and then do one of the other strategies.

[Zannick]

Another option: proportional eviction. We try evicting 1/x of each segment where x is the queue capacity divided by the min evictions times the number of segments. We'd probably want to avoid evicting everything out of a segment as well.