TMM queue pop timer can march to negative infinity

[Sheikah45]

It looks like there is an issue where the pop timer can get into some kind of loop that marches down to 0 then negative infinity and then can never recover.

Server graph attached

Not sure from the logs what caused it but occurred around 17:07 US east on July 27th

[Sheikah45]

The beginning of the end

INFO     Jul 27  20:58:41 PopTimer                       Next tmm3v3 wave happening in 61s
INFO     Jul 27  20:59:43 PopTimer                       Next tmm3v3 wave happening in 58s
INFO     Jul 27  21:00:41 PopTimer                       Next tmm3v3 wave happening in 53s
INFO     Jul 27  21:01:35 PopTimer                       Next tmm3v3 wave happening in 47s
INFO     Jul 27  21:02:22 PopTimer                       Next tmm3v3 wave happening in 43s
INFO     Jul 27  21:03:06 PopTimer                       Next tmm3v3 wave happening in 37s
INFO     Jul 27  21:03:43 PopTimer                       Next tmm3v3 wave happening in 29s
INFO     Jul 27  21:04:13 PopTimer                       Next tmm3v3 wave happening in 24s
INFO     Jul 27  21:04:38 PopTimer                       Next tmm3v3 wave happening in 20s
INFO     Jul 27  21:04:58 PopTimer                       Next tmm3v3 wave happening in 16s
INFO     Jul 27  21:05:14 PopTimer                       Next tmm3v3 wave happening in 13s
INFO     Jul 27  21:05:28 PopTimer                       Next tmm3v3 wave happening in 11s
INFO     Jul 27  21:05:40 PopTimer                       Next tmm3v3 wave happening in 10s
INFO     Jul 27  21:05:51 PopTimer                       Next tmm3v3 wave happening in 9s
INFO     Jul 27  21:06:00 PopTimer                       Next tmm3v3 wave happening in 9s
INFO     Jul 27  21:06:09 PopTimer                       Next tmm3v3 wave happening in 8s
INFO     Jul 27  21:06:18 PopTimer                       Next tmm3v3 wave happening in 7s

[Askaholic]

Looks like this can happen because the way we compute the 'queue rate' is just the number of players in queue over the time since the last pop. This would only be correct if players re-queue each time they fail to find a match. However, since that's not how the queue works, if a large number of players remains unmatched in the queue for a while, the time can start marching down, and once it gets to a critical level it will start marching down very fast.

[Askaholic]

A quick solution would be to add a minimum pop time to prevent that from happening, but the underlying issue is that the 'queue rate' is not calculated correctly.