Support data exceeding memory by using client-side caches compression and eventually redis if needed

[farley13]

really R&D needed here - this is for massive datasets or external datasets