I have a large .bson file (468 MB). After gzipping it to .bson.gz, the file size is reduced to 169 MB (~3x compression). How feasible would it be to have a macro that automatically saves/loads a gz compressed bson file? Or is there a way to do so already using something like CodecZlib.jl?
Interestingly, using the same data I tried an experiment with JLD2 files (see issue), and their file size is 615 MB (.jld2) and 187 MB (.jld2.gz). It seems like .bson has better compression than .jld2, although the loading speed for .jld2 files are 3-4x faster (1.5 sec vs 5 sec). Is this a correct interpretation?
biona001
commented
4 years ago
Hello,
I have a large
.bsonfile (468 MB). After gzipping it to.bson.gz, the file size is reduced to 169 MB (~3x compression). How feasible would it be to have a macro that automatically saves/loads a gz compressed bson file? Or is there a way to do so already using something likeCodecZlib.jl?Interestingly, using the same data I tried an experiment with JLD2 files (see issue), and their file size is 615 MB (.jld2) and 187 MB (.jld2.gz). It seems like
.bsonhas better compression than.jld2, although the loading speed for.jld2files are 3-4x faster (1.5 sec vs 5 sec). Is this a correct interpretation?