Because of central limit theorem the function of the block-size will have a normal distribution, with average being around 32 bytes (and a very small variance), so some buckets will still be too much long compared to the others.
Even if the buckets approach increased a lot the performances, using a fixed number of buckets is not the optimal case as some buckets could still contain more than 50% of the blocks.
A better approach would be to have a bucket for each different size, which in the worst case would require to have a number of checks equals to the number of different sizes. This number of sizes seems always be a very small amount (around 100-150 in core.ypt which is the asset with the highest amount of blocks), but once we have our bucket we can just pick the tail/head item
compared to the actual approach which allows to access a bucket in constant time but the number of elements to iterate in such bucket could be still very high!
So the idea is to use something like a Dictionary<long,LinkedList> for buckets
And have a property to cache keys collection sorted which removes the specific key when a bucket has all its blocks removed.
Because of central limit theorem the function of the block-size will have a normal distribution, with average being around 32 bytes (and a very small variance), so some buckets will still be too much long compared to the others. Even if the buckets approach increased a lot the performances, using a fixed number of buckets is not the optimal case as some buckets could still contain more than 50% of the blocks. A better approach would be to have a bucket for each different size, which in the worst case would require to have a number of checks equals to the number of different sizes. This number of sizes seems always be a very small amount (around 100-150 in core.ypt which is the asset with the highest amount of blocks), but once we have our bucket we can just pick the tail/head item compared to the actual approach which allows to access a bucket in constant time but the number of elements to iterate in such bucket could be still very high!
So the idea is to use something like a Dictionary<long,LinkedList> for buckets And have a property to cache keys collection sorted which removes the specific key when a bucket has all its blocks removed.