calling saveSimList() using the previous dev-stable versions of SpaDES.core and reproducible produced reasonable file sizes when using fileBackEnd = 2 (now deprecated):
rep01$ du -sh simOut*.qs
125M simOutDataPrep_NRR_Cariboo.qs
1.2G simOutPreamble_NRR_Cariboo.qs
46M simOutSpeciesLayers_NRR_Cariboo.qs
using the current development versions (with latest modules using terra) causes the R session to hang (for hours before I killed it), as it seems to be writing massive files: preamble was over 100GB and Biomass_speciesData more than 10GB in size. I am using saveSimList(sim, file, inputs = FALSE, outputs = FALSE, cache = FALSE).
calling
saveSimList()
using the previousdev-stable
versions ofSpaDES.core
andreproducible
produced reasonable file sizes when usingfileBackEnd = 2
(now deprecated):using the current
development
versions (with latest modules using terra) causes the R session to hang (for hours before I killed it), as it seems to be writing massive files: preamble was over 100GB andBiomass_speciesData
more than 10GB in size. I am usingsaveSimList(sim, file, inputs = FALSE, outputs = FALSE, cache = FALSE)
.I'm not sure whether this is related to https://github.com/PredictiveEcology/reproducible/issues/359, since my understanding is that
saveSimList()
uses some of theCache()
infrastructure to deal with file-backed R objects, etc.It may be a qs issue -- debugging this, I manually set the filename to .rds and used
saveRDS()
which wrote the pramble simList to a 2 GB file.