search

qsbase / qs

Quick serialization of R objects
400 stars 19 forks source link

QS cannot compress R6 object efficiently #6

Closed jerryfuyu0104 closed 4 years ago

jerryfuyu0104 commented 5 years ago

RiskModel <- R6Class( classname = "RiskModel", public = list( name = NULL, covMat = NULL, ## N * N initialize = function(name, covMat = NULL) { } ), private = NULL, active = NULL, inherit = NULL, lock_objects = TRUE, class = TRUE, portable = TRUE, lock_class = FALSE, cloneable = TRUE, parent_env = parent.frame() )

if i initialize RiskModel with covMat size:10000*10000, the object serialized by readr::write_rds and qs::qsave will get two files with almost same size. when i serialized covMat directly, the size of two files will be much different!

traversc commented 5 years ago

qs relies on built in R serialization for S4 objects and higher, so if covMat is part of a S4/R6 object, it'll just be serialized by the default function. The reason for this is that I built qs for serializing mainly big data.frames, and nested lists (e.g. JSONs), for now.

It's definitely possible to more efficiently serialize S4/R6 objects, and it is on my (long) to do list :)

traversc commented 4 years ago

Hey this should be more efficient now. Give it a shot if you are still interested :)