search

kedartatwawadi / NN_compression

MIT License
213 stars 42 forks source link

I use GZIP to compress all of 10-50 markovity datasets but there is no different size among them #9

Open viviancui59 opened 6 years ago

viviancui59 commented 6 years ago

Should I try higher markovity dataset to compress using Gzip? The datasets are all generate by your generating code.

viviancui59 commented 6 years ago

I try to use myself code to compress these dataset, but those compressed datasets' sizes are the same with each other, so I try to use GZIP to compress, the results are identical.

sonack commented 6 years ago

How did you use gzip to compress files? Did you compress directly on the generated .txt text files? I am wondering how much the encoding schema will affect the final comparision with NN_compression.

AnsonHooL commented 5 years ago

How did you use gzip to compress files? Did you compress directly on the generated .txt text files? I am wondering how much the encoding schema will affect the final comparision with NN_compression.

How to compute compress ratio by loss fuction？师兄

kedartatwawadi commented 5 years ago

Based on my experiments, gzip should be able to compress markovity 10 dataset much better than markovity 50.

Can you post the command you used for generate the files (with the options).

@AnsonHooL: The loss value is compression ratio on the batch (without accounting for the 2/N factor for arithmetic coding)