This should be the final batch of breaking change to the intermediate format.
Main updates:
Rename to IntermediateColumnarFormat (Pickled chunks is an implementation detail)
Moved some more information into the metadata (chunk size, and compressor)
Changed default compressor to LZ4 (in my quick analysis, this leads to about twice the size of the ICF in about half the compression time. I'll follow up on this separately)
This should be the final batch of breaking change to the intermediate format.
Main updates: