Fix memory cache misses in some cases where precision is not set.

[colinrtwhite]

This is larger refactor that ensure we always query the memory cache and perform decoding with the same sizeResolver, scale, and precision to avoid mismatches.