search

dmlc / XGBoost.jl

XGBoost Julia Package
Other
288 stars 110 forks source link

Compressed Save #147

Closed BabaYara closed 1 year ago

BabaYara commented 1 year ago

I was wondering if compression can be added to the save booster saving method.

ExpandingMan commented 1 year ago

It does not look like there are any compression options in libxgboost, and as such I don't think it's appropriate to add additional dependencies to this library for compression.

There is of course nothing stopping you from compressing the serialized output, see, for example TranscodingStreams.jl for a convenient way of doing this. For example, you can do something like 

open(GzipCompressorStream, "booster.dat.gz", write=true) do io
    XGBoost.save(booster, io)
end

to write the XGBoost model booster to a file using gzip compression.

BabaYara commented 1 year ago

Thanks a lot for the suggestion!

ExpandingMan commented 1 year ago

Glad it helped. I'm going to close this now, but feel free to comment if you feel the interface could use improvement, as it was not really written with compression in mind. Again, I don't think it's a good idea to add a dependency, but there are always possibilities for small API tweaks and improvements.