Running a simulation from a PDA posterior - issues loading posterior files

[serbinsh]

Bug Description

Added posterior from @istfer to our BNL BETY database here: https://modex.bnl.gov/bety/posteriors/1000013118#

Ran this to insert the posterior data into modex BETY:

bety <- list(user='bety', password='bety', host='localhost',
             dbname='bety', driver='PostgreSQL',write=TRUE)
con <- PEcAn.DB::db.open(bety)
bety$con <- con

filename <- '/data/pecan_dbfiles/posterior/1000013118/trait.mcmc.pda.temperate.deciduous.ALL_1000020009.Rdata'
hostname=PEcAn.remote::fqdn()
posteriorid <- '1000013118'
dbfile.insert(in.path = dirname(filename), in.prefix = basename(filename), 
              type = "Posterior", id = posteriorid, con = con, reuse = TRUE, hostname = hostname)

filename <- '/data/pecan_dbfiles/posterior/1000013118/post.distns.pda.temperate.deciduous.ALL_1000020009.Rdata'
hostname=PEcAn.remote::fqdn()
posteriorid <- '1000013118'
dbfile.insert(in.path = dirname(filename), in.prefix = basename(filename), 
              type = "Posterior", id = posteriorid, con = con, reuse = TRUE, hostname = hostname)

But run into this issue trying to use the posterior in the run

Error in readChar(con, 5L, useBytes = TRUE) : cannot open the connection
Calls: get.parameter.samples -> load -> readChar
In addition: Warning messages:
1: In rm(prior.distns, post.distns, trait.mcmc) :
  object 'prior.distns' not found
2: In rm(prior.distns, post.distns, trait.mcmc) :
  object 'post.distns' not found
3: In rm(prior.distns, post.distns, trait.mcmc) :
  object 'trait.mcmc' not found
4: In readChar(con, 5L, useBytes = TRUE) :
  cannot open compressed file '/data/pecan_dbfiles/1000013118/prior.distns.Rdata', probable reason 'No such file or directory'
Execution halted

From @istfer "looks like a bug in get.parameter.samples"

log output

Error in readChar(con, 5L, useBytes = TRUE) : cannot open the connection
In addition: Warning messages:
1: In rm(prior.distns, post.distns, trait.mcmc) :
  object 'prior.distns' not found
2: In rm(prior.distns, post.distns, trait.mcmc) :
  object 'post.distns' not found
3: In rm(prior.distns, post.distns, trait.mcmc) :
  object 'trait.mcmc' not found
4: In readChar(con, 5L, useBytes = TRUE) :
  cannot open compressed file '/data/pecan_dbfiles/1000013118/prior.distns.Rdata', probable reason 'No such file or directory'
> traceback()
7: readChar(con, 5L, useBytes = TRUE)
6: load(file.path(outdirs[i], "prior.distns.Rdata"))
5: get.parameter.samples(settings, ens.sample.method = settings$ensemble$samplingspace$parameters$method) at Multisite_SDA_BNL.R#63
4: eval(ei, envir)
3: eval(ei, envir)
2: withVisible(eval(ei, envir))
1: source("R_scripts/Multisite_SDA_BNL.R")

so its trying to open prior.distns.Rdata instead of post.distns.*Rdata it seems

https://github.com/PecanProject/pecan/blob/494b6717440e0d8245ec9234040f718b807bc29f/modules/uncertainty/R/get.parameter.samples.R#L58
maybe just need to update this to fit the mcmc sampling on the lines below?

Something like?

> filename <- '/data/pecan_dbfiles/posterior/1000013118/post.distns.pda.temperate.deciduous.ALL_1000020009.Rdata'
> list.files(path = dirname(filename), pattern = glob2rx("post.distns.*.Rdata"), full.names=T)
[1] "/data/pecan_dbfiles/posterior/1000013118/post.distns.pda.temperate.deciduous.ALL_1000020009.Rdata"
>

** Note: thread pasted from Slack to preserve it

[istfer]

I can dig deeper at this as of next week (spring break) because I also need to make sure that the workflow finds PDA posteriors properly

Just wanted to note ahead of time that the code also needs to be able to find related trait.mcmc.Rdata files for PDA posteriors, we don't want to sample only from the marginal distributions listed in the post.distns.Rdata files

[serbinsh]

Thanks for adding to the comments @istfer! I just wanted to get this thread started so we can make sure it gets addressed!

[github-actions[bot]]

This issue is stale because it has been open 365 days with no activity.