The current dictionaries were generated by sampling data against our snapshot, which contains realistic data but of less variety than a proper mainnet checkpoint would.
It would probably improve compression further to index a mainnet checkpoint completely and then regenerate the dictionaries, or to change the sample generation for dictionaries to read from checkpoints instead of reading from our index.
Description
The current dictionaries were generated by sampling data against our snapshot, which contains realistic data but of less variety than a proper mainnet checkpoint would.
It would probably improve compression further to index a mainnet checkpoint completely and then regenerate the dictionaries, or to change the sample generation for dictionaries to read from checkpoints instead of reading from our index.