RadioAstronomySoftwareGroup / pyuvsim

A ultra-high precision package for simulating radio interferometers in python on compute clusters.
https://pyuvsim.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
43 stars 7 forks source link

Update Ntasks calc #349

Closed mkolopanis closed 3 years ago

mkolopanis commented 3 years ago

Sum all the task over each node to find total number of tasks.

Description

Changed the Ntasks_tot to multiply ntasks_local * nsky_parts then sum over each rank to get an accurate total number of tasks.

Motivation and Context

The number of reported tasks wasn't exactly accurate in all cases. Especially if one node had less RAM than others.

Types of changes

Checklist:

For all pull requests:

Bug fix checklist:

New feature checklist:

Breaking change checklist:

Reference simulation update or replacement:

Documentation change checklist:

Version change checklist:

Build or continuous integration change checklist:

codecov[bot] commented 3 years ago

Codecov Report

Merging #349 (6bf8d92) into main (d63a968) will increase coverage by 0.00%. The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #349   +/-   ##
=======================================
  Coverage   99.02%   99.02%           
=======================================
  Files          13       13           
  Lines        1943     1945    +2     
=======================================
+ Hits         1924     1926    +2     
  Misses         19       19           
Impacted Files Coverage Δ
pyuvsim/uvsim.py 98.46% <100.00%> (+<0.01%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update d63a968...6bf8d92. Read the comment docs.

mkolopanis commented 3 years ago

the rtd build seems to have given up the last few pushes (excessive memory), but I don't seem to be able to restart it.

mkolopanis commented 3 years ago

@bhazelton any ideas about the RTD? I know you have done some work with the out of memory issues before. What I really don't understand is why this PR is different from #352 and #350 in terms of rtd. Neither of those had build trouble :thinking:

bhazelton commented 3 years ago

I'm not sure what the issue was, but I just re-ran it and it worked. I did notice that the python version is pinned to 3.6 in ci/rtd_env.yml, which isn't great. I think we should change it to at least 3.7 (or 3.8 so it'll last longer).

bhazelton commented 3 years ago

@aelanman can you re-approve?