bazingagin / npc_gzip

Code for Paper: “Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors
MIT License
1.77k stars 155 forks source link

Dynamically import compression module in compressors.py #1

Closed congruency closed 1 year ago

congruency commented 1 year ago

Let the example adapt to any compression algorithm (that conforms to spec) by importing the module under an exception handler.

Example using zstd, which is not found in the original compressors.py

root@b06f53bafe18:/workspace# python main_text.py --compressor gzip
400
400
KNN with compressor=gzip
400it [00:07, 50.78it/s]
Accuracy is 0.6875
spent: 7.89499568939209
root@b06f53bafe18:/workspace# python main_text.py --compressor zstd
400
400
KNN with compressor=zstd
400it [00:02, 172.99it/s]
Accuracy is 0.7375
spent: 2.3285083770751953
Briancarl07 commented 1 year ago

Nice