This PR added the highly modulized block-based SDA workflow, which has complete try-catch detections that will facilitate the user-end debugging process. This workflow is also highly parallelized, which will increase the speed a lot.
Motivation and Context
The multisite SDA workflow has been problematic due to the high-dimension Bayesian sampling and speed. By using the block-based SDA workflow, people can maintain the multi-site structure and, meanwhile, the covariance structure. For the Wishart case of the MCMC sampling, it's been problematic due to the changes in observations, which is now been solved by employing the global aqq that is defined by the number of state variables.
Beyond that, this PR has the following updates:
1) Allow pre-existed aqq & bqq imported from outside.
2) Allow pre-specified prior of Q imported from outside.
3) Enabled the MCMC sampling with NA observations to help estimate the process variance for a free run.
4) The qsub_parallel function is updated and more robust.
5) Fixed the bug of the edge cases of the single site and single observation scenario.
6) Fixed the bug where there is no nc_close and on.exit() paired with each nc_open function for the prepare_pools function.
Review Time Estimate
[ ] Immediately
[ ] Within one week
[x] When possible
Types of changes
[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
Checklist:
[x] My change requires a change to the documentation.
Description
This PR added the highly modulized block-based SDA workflow, which has complete
try-catchdetections that will facilitate the user-end debugging process. This workflow is also highly parallelized, which will increase the speed a lot.Motivation and Context
The multisite SDA workflow has been problematic due to the high-dimension Bayesian sampling and speed. By using the block-based SDA workflow, people can maintain the multi-site structure and, meanwhile, the covariance structure. For the Wishart case of the MCMC sampling, it's been problematic due to the changes in observations, which is now been solved by employing the global
aqqthat is defined by the number of state variables. Beyond that, this PR has the following updates: 1) Allow pre-existed aqq & bqq imported from outside. 2) Allow pre-specified prior of Q imported from outside. 3) Enabled the MCMC sampling with NA observations to help estimate the process variance for a free run. 4) Theqsub_parallelfunction is updated and more robust. 5) Fixed the bug of the edge cases of the single site and single observation scenario. 6) Fixed the bug where there is nonc_closeandon.exit()paired with eachnc_openfunction for theprepare_poolsfunction.Review Time Estimate
Types of changes
Checklist: