search

LSSTScienceCollaborations / ObservingStrategy

A community white paper about LSST observing strategy, with quantifications via the the Metric Analysis Framework.
59 stars 75 forks source link

End-to-end Figure of Merit examples for the Science Collaborations #480

Open willclarkson opened 8 years ago

willclarkson commented 8 years ago

Info request: recommended IPython maf tutorials for science collaboration members?

One of the goals of the July 13th web meeting of the Stars, Milky Way and Local Volume (SMWLV) collaboration, is to help collaboration members learn how to translate their science goals into figures of merit (FoMs) for the Observing Strategy Whitepaper. Talking to some of the SMWLV members, there still seems to be a gap between coming up with science goals and encoding them into FoMs for sims_maf.

To try to close that gap and provoke input from collaboration members, I plan to show one or two end-to-end examples of FoM construction at the SMWLV web meeting on the 13th - ideally, showing an ipython notebook producing the FoM that appears in the whitepaper.

Many of the FoMs presented in the Galaxy chapter are in the repository lsstScratchWIC (in the "notebooks" directory), but those are fairly rudimentary and a little bit cargo-cult - there must be better examples.

@yoachim and @rhiannonlynne - are there any Tutorial examples in the maf or maf_contrib repository that correspond to FoMs in the whitepaper already? If not, are there any examples you'd recommend presenting to SMWLV?

@AshishMahabal , @lmwalkowicz , @fedhere, @ebellm, @dnidever, @knutago, @jasondrhodes, @lundmb - are there examples of FoM calculations (preferably IPython notebooks) that you'd recommend presenting to an interested collaboration member who might have an idea for a FoM that needs translation to sims_maf?

cc @akvivas cc @jgizis

drphilmarshall commented 8 years ago

@willclarkson How did the July 13th SMWLV meeting go in the end? Did you show any FoM examples in notebook form?

willclarkson commented 8 years ago

Hi @drphilmarshall - I believe @jgizis was going to circulate his notes from that meeting, I don't know if that's happened. Here's a quick read:

  1. @jgizis has set up and circulated a brief survey for the SMWLV collaboration, to get yes/no input on whether the current WP version adequately covers their science cases, and if not, what's missing;
  2. We made another plea to SMWLV to contribute to the WP effort - to code science cases, or, at the least, describe the science cases to the point at which they might be specified in the WP;
  3. As an example, I showed the notebook for the "Galactic Supernova" figure of merit, which can be found at my unofficial LSST scratch repo;
  4. There was helpful discussion (by Josh Pepper and others) about building figures of merit from the "lego pieces" of the metrics already developed. The "Transients & Variables" collaboration subgroup has been active developing metrics;
  5. There was some concern over Deep Drilling fields; as we well know, some science cases (particularly certain kinds of variables) can probably only be done as DDFs;
  6. @ctslater gave a nice update on his progress towards improving star/galaxy separation, which will inform several pieces of the WP. Comparison of existing S/G separation metrics has been progressing. There was discussion about ways one might translate this into science figures of merit (e.g. is specifying a desired level of contamination useful or feasible?). It's a complex subject.

I think that covers most of what went on in the meeting. @jgizis, if I have missed anything, please comment!