nanograv / enterprise

ENTERPRISE (Enhanced Numerical Toolbox Enabling a Robust PulsaR Inference SuitE) is a pulsar timing analysis code, aimed at noise analysis, gravitational-wave searches, and timing model analysis.
https://enterprise.readthedocs.io
MIT License
65 stars 67 forks source link

Add partially shared Pulsar objects #289

Closed vallis closed 3 years ago

vallis commented 3 years ago

This PR adds Pulsar methods deflate(), inflate(), and destroy() that allow sharing Pulsar objects among processes on the same host/node.

The workflow is as follows:

This functionality requires Python 3.8 (so it is not tested in 3.6 and 3.7). It also requires dedicated testing on our production clusters.

codecov[bot] commented 3 years ago

Codecov Report

Merging #289 (e92596b) into master (9ad4a29) will increase coverage by 0.19%. The diff coverage is 98.52%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #289      +/-   ##
==========================================
+ Coverage   86.76%   86.95%   +0.19%     
==========================================
  Files          12       13       +1     
  Lines        2819     2868      +49     
==========================================
+ Hits         2446     2494      +48     
- Misses        373      374       +1     
Impacted Files Coverage Δ
enterprise/pulsar_inflate.py 96.15% <96.15%> (ø)
enterprise/pulsar.py 91.96% <100.00%> (+0.50%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 9ad4a29...e92596b. Read the comment docs.

vallis commented 3 years ago

Coverage fails on 3.6 and 3.7 because of the new shared-memory code. Should be able to exclude it along the lines of https://sobolevn.me/2020/02/conditional-coverage

AaronDJohnson commented 3 years ago

I have tested this branch with both multiple serial tasks running on the same node and with MPI tasks running on the same node on a local cluster. Both versions are working as intended when using a Slurm scheduler.