Currently, the webppl team is investigating using callbacks to support streaming samples to disk. Once this pull request is merged, we should investigate defaulting to this option (with the additional onlyMAP: true option for inference methods) to incrementally stream the samples to file (e.g., when user is using MCMC, incrementalMH, SMC). We already write the samples to file, so this will just incrementalize the process and save lots of memory.
Currently, the webppl team is investigating using callbacks to support streaming samples to disk. Once this pull request is merged, we should investigate defaulting to this option (with the additional
onlyMAP: trueoption for inference methods) to incrementally stream the samples to file (e.g., when user is using MCMC, incrementalMH, SMC). We already write the samples to file, so this will just incrementalize the process and save lots of memory.