Possible Compression Improvements

seung-lab / crackle

3D dense segmentation compression codec based on crack codes.

BSD 3-Clause "New" or "Revised" License

6 stars 0 forks source link

Possible Compression Improvements #5

Open william-silversmith opened 1 year ago

william-silversmith commented 1 year ago

[x] Experiment more with pins
[x] Relative coding of chains to increase letter frequency
[x] Code beginning of chain (BOC) using rows
[ ] Reduce BOC markers by allowing a different color pen to connect connected components
[ ] Allow tracing around the edge of the image if it would eliminate enough BOC
[ ] 3D motion prediction for crack codes (hard!)
[ ] Hexagonal pixels? (1 bit per a direction change, similar path length, https://art-of-logic.blogspot.com/2014/12/hexagonal-grid-city-life.html)

william-silversmith commented 1 year ago

I tried the BOC optimization, but the image statistics are such that not too many chains start from the same row and so the overheads made it so that the optimized version was ~0.5% bigger. Maybe this could be optimized by picking the starts to be along a row.

You run into the set cover problem again. :P

william-silversmith commented 1 year ago

Managed to improve pins such that on connectomics.npy, I am seeing a 4x reduction in the amount of data pre-gzip. After gzip? A 7% improvement. Pins are already slow, but a better optimizer than greedy might result in further substantial gains (at high compute cost).

william-silversmith commented 1 year ago

BOC are pretty small... ~2% in connectomics.npy. Marginal improvements possible even though there may be substantial gains on that 2%.