finish implementation of annular selection in configs

[mcclearyj]

In the shear-profiles module, rather than having hard-coded lensing catalog selections, pass a configuration file (superbit_lensing/shear_profiles/configs/default_annular_config.yaml) with reasonable values that can be adapted for different clusters if need be.

Somewhat related: there has been some back-and-forth regarding where to collect assortedgalsim and medsmaker-specific configuration files, but for now, I think we are keeping them in sub-directories called config within the appropriate module directory. As such, the location of configs for superbit_lensing/shear_profiles/make_annular_catalog.py follows our current convention.

[sweverett]

Sorry to drop in randomly after ages, but wasn't that something I added a while back? Was that on some lost branch that never got merged in?

[mcclearyj]

Oh jeez,

Sorry to drop in randomly after ages, but wasn't that something I added a while back? Was that on some lost branch that never got merged in?

Oh jeez, that would be very unfortunate indeed. This process-real branch has been evolving to capture some of the nuances of analyzing the real data (mostly at the medsmaking stage), so it's possible that if your making the shear selections configurable was implemented in another branch, it never made it into process-real.

[sweverett]

For this specifically that's fine, I can dig up the code or outline my procedure if you haven't done it already.

The larger issue is that oba should really be merged in at some point, and that could be a disaster

[sweverett]

Short version: I defined a few different types of selections (equality, inequality, composite, etc.). Each of those were subclasses of some Selection class or something like that. Each instance needs to know (1) type of selection, (2) column of selection quantity, value(s), etc. Composite was more complicated. Then it's easy to make a config layer that would look something like the following:

selection:
  - {type: inequality, col: z, range: (.5, 2), strict: false}
  - {type: equality, col: class, value: galaxy}
  - ...

(too lazy to write it out fully)

[sweverett]

The nice part is that it is then easy to automate say 10 runs with different selections to see how impacted your results are

[GeorgeVassilakis]

Hi Spencer! I hope you're doing well :). I totally agree that oba should be integrated. We were discussing a potential full rebuild of the pipeline once we were done with the initial processing of some representative clusters. There is a lot of great code (with especially robust, well-documented coding practice) in oba. So, if we do this rebuild in the future, we would most likely use oba as a starting point and merge in our updates from process-real, along with relevant forecasting/galsim modules.

Also, that annular selection cut exercise would be amazing. Is that class nested in oba? If so, I'll take a look to see if we can hack it into process-real to do some quick tests before we'd start developing a huge, new merger pipeline.

[mcclearyj]

Chiming in to second @GeorgeVassilakis's statements about merging OBA, or even better, using it as a starting point for a new SuperBIT lensing analysis pipeline. I know OLA is supposed to be doing the same but since they've moved away from ngmix/metacal for shear measurement, there would be value in having an OBA++ pipeline too.

[sweverett]

RE rebuilding from scratch, I used OBA & the pipeline code in this repo as the initial scaffolding of a fancier package called chimera that is being developed primarily for the SPHEREx cosmology pipeline but i wrote such that it can be used generically. For the moment it's housed in a SPHEREx private repo, but i'm going to move it to my own repo in the nearish future. Maybe y'all would be interested in that

[sweverett]

It's got the features you've come to know and love (hopefully lol) but a lot of additional things like parsl support and allowing for different environments for each module, if necessary. And a more generic version of the IOManager