I moved where we submit total packet size so it's right against sending the bytes. This presents fewer opportunities to miss adding the bytes because I noticed that DTLS handshakes were never added to the total. There is one spot in server (commented) where it will won't be accurate, but it is an improvement.
I made PacketPool non-static. If a single user were to cause a flood of packets, those packets would never be released before. Now the packet pools are isolated and released when a user disconnects. Should reduce baseline memory pressure albeit at the cost of more allocations as players connect. Also something to be said for less potential for cross-contamination or shared-resource bugs.
I delayed starting the KeepAlive thread on clients until the connection is established. Unlikely to make a huge difference, but there is no good reason to send a ping if a connection hasn't even been established yet. The server ideally would just toss those packets anyways.