Compression algorithms and packing - Githubissues

jahwag / ClaudeSync

ClaudeSync is a Python tool that automates the synchronization of local files with Claude.ai Projects

MIT License

165 stars 24 forks source link

Compression algorithms and packing #59

Closed jahwag closed 2 weeks ago

jahwag commented 2 weeks ago

Experimental features for compressing project context window size.

Claude doesn't support anything but "pack" and possibly short files using "rle", "dictionary". But they're inefficient.
"pack" is very fast - far faster than syncing individual files. I use this. Leaving default untouched.

Compression Algorithm Comparison

Original Project Size: 50.80 KB

Algorithm	Compressed Size (KB)	Compression Factor	Notes
RLE	446.70 KB	0.11x	Not suitable for non-repetitive data, significantly increases file size.
Huffman	44.35 KB	1.15x	Modest compression, effective only for specific data types.
Dictionary	40.25 KB	1.26x	Simple, fast compression with limited effectiveness.
bz2	9.39 KB	5.41x	Good compression, slightly slower than zlib.
zlib	9.24 KB	5.49x	Balanced choice with good compression and speed.
LZMA	8.76 KB	5.80x	Best compression, but slower to process.

Note. Compression factor increases on larger projects.