search

msimet / Stile

Stile: the Systematics Tests In Lensing pipeline
BSD 3-Clause "New" or "Revised" License
9 stars 6 forks source link

Parallelization #97

Open msimet opened 7 years ago

msimet commented 7 years ago

Eventually we will have too much data to run some of these plots on one processor. We should start thinking about ways to parallelize a lot of these tests.

msimet commented 6 years ago

Noting for the curious that I did most of this work on branch #115 (which updates how we return results--a necessary change, I think, to make this work). Or--to be precise, I allowed us to accumulate results from chunks of a catalog. Parallelization is still kind of beyond us for anything but the StatSysTest, since running on a later chunk depend on choices from previous chunks.

To make everything run including unit tests, I merged some various other branches into that one, and I will eventually need to do the BinnedScatterPlot and BinnedWhiskerPlot as well, so I'll wait on a PR for this until the PRs for the dependent branches & the Binned tests are done.