Normal Galactic Plane cadence evaluation discussion

[drphilmarshall]

@willclarkson @akvivas @caprastro @bethwillman

Hi Galaxy Fans! I just saw in #162 that @rhiannonlynne finished the new OpSim run with the galactic plane observed at "normal" cadence, as per Jay's suggestion. While I work on auditing your white paper sections, we might like to start discussing how and when to analyze this data, and also how to publish the results. I'm assuming this would be a summer project, the results of which we could potentially discuss in August in Tucson.

I would think that evaluating the impact of this new strategy on a variety of other science cases as well as your own would be a very important part of the analysis. in an ideal world, we'd run all of our Figures of Merit on it, and make a comprehensive statement about the proposed strategy. Maybe this could be a good test case for attempting such an automated analysis, in a simple baseline vs. The Galaxy bake-off?

[willclarkson]

Thanks @drphilmarshall - my two cents is that it makes sense to rerun the Figures of Merit on this new run as well and add its column to the summary tables in Chapter 4.

Before doing so, however, I would like us to implement just a few more of the figures of merit currently described in that chapter. There is now a sort of pseudocode for several of the FoMs (particularly in the Disk section), which probably should be reasonably straightforward to actually run. With the Tucson meeting still of order two months away, I think there is time to implement a few more FoMs before rerunning on the latest OpSim runs.

I believe @rhiannonlynne made some edits to the configuration of the Plane proposal run in order for it to run by this date, so I don't remember if it's entirely what @caprastro originally proposed.

cc @akvivas @bethwillman @cbritt4 @caprastro @chomiuk @ctslater

-- Will

[rhiannonlynne]

It was actually what was proposed, in the end -- WFD cadence where the galactic plane proposal was within the dec limits of the original WFD survey, and original 'galactic plane' cadence for the little spur in the north and to the south of the WFD region.

[willclarkson]

Lynne - great, thank you!!

Will

Dr. Will Clarkson Assistant Professor of Physics and Astronomy University of Michigan-Dearborn

On Wed, Apr 27, 2016 at 11:09 PM, Lynne Jones notifications@github.com wrote:

It was actually what was proposed, in the end -- WFD cadence where the galactic plane proposal was within the dec limits of the original WFD survey, and original 'galactic plane' cadence for the little spur in the north and to the south of the WFD region.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/355#issuecomment-215298123

[drphilmarshall]

@willclarkson I think I heard at today's meeting ( #293 ) that you would run the metrics that you already have on astro_lsst_01_1004 and that would be that for Version 1 of the white paper. Is this something you can do on a timescale of days, or by mid-May? The question is, when to start the internal review of the Disk section.

If analysis of astro_lsst_01_1004 is going to appear in Version 1, then Chapter 2 will need updating to include a description of it (and perhaps some basic metric analysis). Is this also something we can get done by mid May?

[rhiannonlynne]

The basic (standard) MAF analysis of astro_lsst_01_1004 is already done: http://astro-lsst-01.astro.washington.edu:8080/metricSelect?runId=3 and http://astro-lsst-01.astro.washington.edu:8080/metricSelect?runId=4 although we do have to add this to the runs available from ops2.

On Thu, Apr 28, 2016 at 10:58 AM Phil Marshall notifications@github.com wrote:

@willclarkson https://github.com/willclarkson I think I heard at today's meeting ( #293 https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/293 ) that you would run the metrics that you already have on astro_lsst_01_1004 and that would be that for Version 1 of the white paper. Is this something you can do on a timescale of days, or by mid-May? The question is, when to start the internal review of the Disk section.

If analysis of astro_lsst_01_1004 is going to appear in Version 1, then Chapter 2 will need updating to include a description of it (and perhaps some basic metric analysis). Is this also something we can get done by mid May?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/355#issuecomment-215510949

[willclarkson]

Hi @drphilmarshall cc @akvivas - there are two related issues from the discussion, so just to ensure I'm reasonably clear:

1. re-running the Galactic Supernova FoM for astro_lsst_01_1004 should be straightforward, and I will work on that unless I hear otherwise. If however we feel that there will not be time for the authors of Chapter 2 to add the accompanying discussion about that run into Chapter 2 by mid-May, then I can certainly hold off. (I would want to retain the column for that run in the summary Tables in 4.2 and 4.3 though.)
2. Other FoMs from Chapter 4: the other FoM I want to implement and run soon is 1.1-1.4 in the Astrometry chapter. Those are sums over the standard metrics for select spatial regions (which, as @rhiannonlynne just pointed out, are available online). If I can get that done by Friday then I plan to include those evaluation values in the relevant Table. Ditto for mid-May.

Thanks!! -- Will

[willclarkson]

Hi @drphilmarshall - it looks like I will be able to get the re-run of the Galactic Supernova done for astro_lsst_01_1004 by early afternoon Friday, then would need to insert it into the chapter. If that's too late for the v1 milestone, this issue could be punted to the v2 Tucson milestone.

[drphilmarshall]

Sounds good, Will - thanks for hustling! I will wait until I see all Galaxy-labelled issues closed or punted before assigning a reviewer.

Thanks for the pointers to the std analysis, Lynne! What would you tweet as being the major impact of the extra galactic plane coverage?

On Thu, Apr 28, 2016 at 3:01 PM, Will Clarkson notifications@github.com wrote:

Hi @drphilmarshall https://github.com/drphilmarshall - it looks like I will be able to get the re-run of the Galactic Supernova done for astro_lsst_01_1004 by early afternoon Friday, then would need to insert it into the chapter. If that's too late for the v1 milestone, this issue could be punted to the v2 Tucson milestone.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/355#issuecomment-215576040

[rhiannonlynne]

Well, you get about the same number of total visits in the overall survey, and you get about the same slew time. Probably because of how the scheduler in opsim prioritizes things, observations for other proposals don't seem affected very much.
However, you do get slightly fewer visits per 'WFD' field in the NormalGalacticPlane run -- understandable because you're including more fields in the 'WFD' region. The typical number of visits per field (in all filters) went from 911 in the baseline to 860 in the NormalGalacticPlane. The final coadded depths are shallower by between 0.03-0.05 magnitudes, depending on band. Visits in the galactic plane region are acquired with tighter airmass limits in the NormalGalacticPlane run (because the Universal proposal has tighter constraints than the standard GalacticPlane proposal), so the visits acquired in the galactic plane might have better seeing distributions (it's hard to separate that out from the standard analysis results).

So: other proposals ambivalent, but WFD proposal in general has fewer visits and shallower coadded depth per field. Galactic plane fields obviously go a lot deeper and get more visits (~860 total instead of ~180), and probably have a better seeing distribution. Whether the impact to the WFD fields is significant to science or not is left to metrics tailored to science (I would have guessed a 5% drop number of visits was not so much of an issue, but the corresponding 0.04 reduction in coadded depth in i band seems like it might be problematic?).

[willclarkson]

Thanks @rhiannonlynne !! For what it's worth, the other test I am trying to get finished at the moment is to break down the standard Astrometry metrics by spatial region from pre-computed metrics (and therefore evaluate Astrometry Figures of Merit 1.1-1.4). That'll provide an estimate of the impact on astrometry to the plane and to non-plane regions directly.

Thanks to help from @yoachim I think I have a good lead on how to do this, and am working on a Notebook to implement it. I don't know if this will be completed by Friday afternoon, but I am optimistic that I can get it run on minion_1016, minion_1020 and the WFD-plane run by mid-May.

It sounds like the reduced depth per WFD field is something that the figures of merit will show quite clearly (in the cases where the reduced depth does make a difference).

cc @akvivas @caprastro

[drphilmarshall]

Excellent - very interesting, thanks Lynne! cc @egawiser

On Thu, Apr 28, 2016 at 5:51 PM, Will Clarkson notifications@github.com wrote:

Thanks @rhiannonlynne https://github.com/rhiannonlynne !! For what it's worth, the other test I am trying to get finished at the moment is to break down the standard Astrometry metrics by spatial region from pre-computed metrics (and therefore evaluate Astrometry Figures of Merit 1.1-1.4). That'll provide an estimate of the impact on astrometry to the plane and to non-plane regions directly.

Thanks to help from @yoachim https://github.com/yoachim I think I have a good lead on how to do this, and am working on a Notebook to implement it. I don't know if this will be completed by Friday, but I am optimistic that I can get it run on minion_1016, minion_1020 and the WFD-plane run by mid-May.

It sounds like the reduced depth per WFD field is something that the figures of merit will show quite clearly (in the cases where the reduced depth does make a difference).

cc @akvivas https://github.com/akvivas @caprastro https://github.com/caprastro

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/355#issuecomment-215603741

[willclarkson]

I have implemented Astrometry Figures of Merit 1.1-1.4 (proper motion and parallax uncertainty for the inner plane and main-survey respectively) to allow direct comparison of the impact of better plane coverage on the rest of the survey. It turns out to be fairly low, although more work is needed.

Table 4.7 values have been filled in for Baseline, PS1 and WFD-plane, a figure has been added showing those selection regions, and a para of discussion has been added to MW_Astrometry.tex .

For interest @rhiannonlynne @akvivas @caprastro @dgmonet @jgizis @yoachim

[willclarkson]

The Galactic Supernova figure of merit has also been evaluated for WFD-plane (astro_lsst_01_1004), and the results included in section 4.2.3 and Table 4.6. This is probably a fairer comparison to Baseline than the PS1 cadence because it visits the same regions of the sky as Baseline (minion_1020 does not try to include dec < -60 or so).

[willclarkson]

Hi @rhiannonlynne and @drphilmarshall -- where did we land on including astro_lsst_01_1004 in the whitepaper? From what I read above I think this would require: 1. adding an \opsimdbref definition for astro_lsst_01_1004, and 2. Including some words in Chapter 2 to describe this run. Are those changes likely to happen by mid-May?

[rhiannonlynne]

I can do that by mid-May, given that you have gotten so far evaluating the other implications for astrometry and galactic supernova.

At some point we need to think how to make sure that everyone runs their relevant metrics on all of the runs under consideration (probably this needs to be something taken on by the MAF crew at UW, once everything is well defined with metric to be run).

(@drphilmarshall should I change milestone & assignee on this issue, or make new issue, or ?)

[drphilmarshall]

Thanks Lynne! Let's re-assign this to "Submit to LSST Pub Board" but try to have it done by mid May, as you say. Really glad to see this going in! :-)

On Mon, May 2, 2016 at 11:20 AM, Lynne Jones notifications@github.com wrote:

I can do that by mid-May, given that you have gotten so far evaluating the other implications for astrometry and galactic supernova.

At some point we need to think how to make sure that everyone runs their relevant metrics on all of the runs under consideration (probably this needs to be something taken on by the MAF crew at UW, once everything is well defined with metric to be run).

(@drphilmarshall https://github.com/drphilmarshall should I change milestone & assignee on this issue, or make new issue, or ?)

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/355#issuecomment-216317353

[rhiannonlynne]

Written into chapter 2 - https://github.com/LSSTScienceCollaborations/ObservingStrategy/pull/412

[willclarkson]

Thanks for adding this @rhiannonlynne !! I will update the relevant parts of the Galaxy chapter with the new opsimdbref during the coming week.

cc @akvivas

[willclarkson]

This seems to have been left open since last year, closing.