New MAF metric: Is LSST at, above, or below the crowding limit at any given point on the sky?

[rblum5]

We will add a metric to the MAF to produce a map of the sky that shows whether the survey has reached the crowding (stellar confusion) limit. The inputs for the MAF are 1) a map of the surface brightness on the sky at any point and 2) a luminosity function(s). Our code will pull image quality data as a function of time and wavelength to compute whether any particular spot in the survey has reached the crowding limit.

[michaelstrauss]

What, in practice, is your definition of the crowding limit?

On Aug 21, 2015, at 5:52 PM, rblum5 notifications@github.com wrote:

We will add a metric to the MAF to produce a map of the sky that shows whether the survey has reached the crowding (stellar confusion) limit. The inputs for the MAF are 1) a map of the surface brightness on the sky at any point and an luminosity function(s). Our code will pull image quality data as a function of time and wavelength to compute whether any particular spot in the survey has reached the crowding limit.

� Reply to this email directly or view it on GitHub.

[knutago]

So it means the point at which confusion noise limits the S/N that you would like to achieve. For stellar applications, we often would like to get to S/N=10, i.e. 10% photometric error. In the confusion limit, the surface brightness fluctuations from the background are equal to 10% of the luminosity of the star that you would like to measure.

[rblum5]

Michael, see here: http://adsabs.harvard.edu/cgi-bin/nph-bib_query?bibcode=2003AJ....126..452O.

[StephenRidgway]

Given observations with a range of image quality, do the higher resolution observations give source identification and positions that aleviate crowding effects for poorer seeing images?

When detecting sources by image differencing, do the same crowding limits apply?

Steve

On Aug 21, 2015, at 3:01 PM, rblum5 notifications@github.com wrote:

Michael, see here: http://adsabs.harvard.edu/cgi-bin/nph-bib_query?bibcode=2003AJ....126..452O http://adsabs.harvard.edu/cgi-bin/nph-bib_query?bibcode=2003AJ....126..452O.

— Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/76#issuecomment-133576105.

[bethwillman]

We really need a MAF tool that returns the source density as a function of magnitude and position on the sky, so that Knut's parameterization of photometric measurement uncertainty as f(density, seeing) can be implemented into metric calculation.

Is that something that will naturally come from this metric? It doesn't matter if the Galaxy model used is perfect - if we get this coded up structurally, then we could plug and play between Besancon, Juric model, and whatever better model we learn from the ongoing generation of DECam surveys.

On Fri, Aug 21, 2015 at 3:01 PM, rblum5 notifications@github.com wrote:

Michael, see here: http://adsabs.harvard.edu/cgi-bin/nph-bib_query?bibcode=2003AJ....126..452O .

— Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/76#issuecomment-133576105 .

Beth Willman AURA/LSST Deputy Director Associate Astronomer, Steward Observatory 933 North Cherry Avenue Tucson, Arizona 85721 520-318-8473

[lundmb]

Whiles it's not the best, I do have a couple things I've added to mafContrib that are intended to use stellar density values. The metrics themselves are CountMetric and CountMassMetric, and they use functions that are in the directory StarCounts. The two metrics look at stars in a range of distances and in a range of masses with sufficiently low noise, respectively, but the functions they're relying on will allow for finding the number of stars in a particular field.

The basic functions convert from equatorial to galactic coordinates, and also calculate stellar density as a function of galactic cylindrical coordinates. The model is Juric 2008 crudely combined with Jackson 2002 to include the bulge. This won't address anything outside the galaxy, however. Also see section 3.3 of this paper: http://arxiv.org/abs/1508.03175

[knutago]

This is perfect! Thanks!

Knut

On Aug 21, 2015, at 3:24 PM, lundmb notifications@github.com wrote:

Whiles it's not the best, I do have a couple things I've added to mafContrib that are intended to use stellar density values. The metrics themselves are CountMetric and CountMassMetric, and they use functions that are in the directory StarCounts. The two metrics look at stars in a range of distances and in a range of masses with sufficiently low noise, respectively, but the functions they're relying on will allow for finding the number of stars in a particular field.

The basic functions convert from equatorial to galactic coordinates, and also calculate stellar density as a function of galactic cylindrical coordinates. The model is Juric 2008 crudely combined with Jackson 2002 to include the bulge. This won't address anything outside the galaxy, however. Also see section 3.3 of this paper: http://arxiv.org/abs/1508.03175 http://arxiv.org/abs/1508.03175 — Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/76#issuecomment-133579213.

[knutago]

Hi Steve,

So I think the first question is a question for DM, but I’m pretty sure the answer is yes.  What I don’t know how to do is to calculate how much a poorer seeing image adds to the depth in the confusion limited regime, but maybe someone else can help.

For difference imaging, having a crowded field just adds some extra Poisson noise, I think, assuming there aren’t systematic residuals.  So it would just reduce the single exposure S/N.

Knut

On Aug 21, 2015, at 3:13 PM, StephenRidgway notifications@github.com wrote:

Given observations with a range of image quality, do the higher resolution observations give source identification and positions that aleviate crowding effects for poorer seeing images?

When detecting sources by image differencing, do the same crowding limits apply?

Steve

On Aug 21, 2015, at 3:01 PM, rblum5 notifications@github.com wrote:

Michael, see here: http://adsabs.harvard.edu/cgi-bin/nph-bib_query?bibcode=2003AJ....126..452O http://adsabs.harvard.edu/cgi-bin/nph-bib_query?bibcode=2003AJ....126..452O.

— Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/76#issuecomment-133576105.

— Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/76#issuecomment-133577724.

[knutago]

Just to summarize where this issue is at Friday evening:

We're defining the confusion limit as the flux at which fluctuations in the astronomical surface brightness exceed that set by the requirement on the S/N. The plan is take the observations in the OpSim database, use the seeing, the astronomical surface brightness at the given location, and a model luminosity function along the line of sight to compute the confusion noise (and limit) for a given observation. We will then write a function to coadd the depths, accounting for the confusion noise, to get the observed limiting depth at any given location in the Galactic Plane and Magellanic Clouds.

We will then produce two metrics. The first is a map of the achieved depth, accounting for the confusion. The second is the number of observations that were needed to reach the confusion limit.

This calculation needs a few inputs:

1. A map of the surface brightness from astronomical sources.
2. A map containing the model of luminosity function at each location. We will get these by modifying Mike Lund's StarCount metric and its supporting functions.
3. The seeing of the observations, from OpSim.
4. The relationship between confusion limit and the above inputs, which we will get from Olsen, Blum, & Rigaut 2003, AJ, 126, 452:

[StephenRidgway]

Nice

On Aug 21, 2015, at 11:37 PM, knutago notifications@github.com wrote:

Just to summarize where this issue is at Friday evening:

We're defining the confusion limit as the flux at which fluctuations in the astronomical surface brightness exceed that set by the requirement on the S/N. The plan is take the observations in the OpSim database, use the seeing, the astronomical surface brightness at the given location, and a model luminosity function along the line of sight to compute the confusion noise (and limit) for a given observation. We will then write a function to coadd the depths, accounting for the confusion noise, to get the observed limiting depth at any given location in the Galactic Plane and Magellanic Clouds.

We will then produce two metrics. The first is a map of the achieved depth, accounting for the confusion. The second is the number of observations that were needed to reach the confusion limit.

This calculation needs a few inputs:

1. A map of the surface brightness from astronomical sources.
2. A map containing the model of luminosity function at each location. We will get these by modifying Mike Lund's StarCount metric and its supporting functions.
3. The seeing of the observations, from OpSim.

4. The relationship between confusion limit and the above inputs, which we will get from Olsen, Blum, & Rigaut 2003, AJ, 126, 452:

   https://cloud.githubusercontent.com/assets/6912765/9422881/6f02eb84-485d-11e5-9aeb-896dc36c2691.gif — Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/76#issuecomment-133644737.

[knutago]

Saturday morning update: we're still working on implementing these metrics in MAF. But to give some idea of where this is heading, here is a plot generated at the LSST2014 Cadence Workshop: Which shows the limiting magnitude for crowding under two different assumptions of seeing as a function of LMC radius.

[rhiannonlynne]

Tagging myself in this issue, because I want to be sure I track the metrics which may be using Mike's metrics.
There are some minor updates I'd like to make to his metrics, after discussing that with him, but in particular 'CountMetric' is not a great name for us to be using since it's a duplicate of a metric name in sims_maf (yes, this is completely nit-picky, but names become important for code maintenance/usability as we go further into the future). @rhiannonlynne (I'm going to tag Peter too .. @yoachim)

(I also think we may want to incorporate the star counts as an available 'map' for MAF, in a similar way as we provide a dust map -- this would make the information more easily available to all metrics).

[lundmb]

I discovered that there's an inbuilt name there, and I think both of the metrics can be modified to just prefix them with 'Star' and avoid the issues of overlapping names. I will also note that I think the map usage may be quite useful, but I'd differentiate between the two potential uses of the star counts; I wrote the metric initially to provide star counts for targets, where a fixed map would not be useful, but I can certainly see the benefits of also having a background source map, along the lines of how this would be useful for crowding.

On Sat, Aug 22, 2015 at 11:49 AM, Lynne Jones notifications@github.com wrote:

Tagging myself in this issue, because I want to be sure I track the metrics which may be using Mike's metrics.

There are some minor updates I'd like to make to his metrics, after discussing that with him, but in particular 'CountMetric' is not a great name for us to be using since it's a duplicate of a metric name in sims_maf (yes, this is completely nit-picky, but names become important for code maintenance/usability as we go further into the future). @rhiannonlynne https://github.com/rhiannonlynne

(I also think we may want to incorporate the star counts as an available 'map' for MAF, in a similar way as we provide a dust map -- this would make the information more easily available to all metrics).

— Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/76#issuecomment-133739670 .

[drphilmarshall]

Hi all! What's the status of this metric, then? Do you have code in sims_maf_contrib to point to? And which sections of the white paper will be using it, do you know? If you think you have achieved your initial goal, please do close this out! Thanks :-)

[rblum5]

Hi Phil,

We are not done. I will talk with Knut and David about this next week.

-Bob

On Wed, Nov 11, 2015 at 5:41 PM Phil Marshall notifications@github.com wrote:

Hi all! What's the status of this metric, then? Do you have code in sims_maf_contrib to point to? And which sections of the white paper will be using it, do you know? If you think you have achieved your initial goal, please do close this out! Thanks :-)

— Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/76#issuecomment-155958432 .

[knutago]

Progress report: I now have a model luminosity function from Besancon for every field in enigma_1189. With this, plus LFs for the Magellanic Clouds and a background galaxy luminosity function, and the inputs should be all there. Might have something preliminary working by Thursday.

[rhiannonlynne]

@yoachim is also working on a stellar luminosity function map I think, tagging him here.

On Tue, Nov 17, 2015, 10:30 AM knutago notifications@github.com wrote:

Progress report: I now have a model luminosity function from Besancon for every field in enigma_1189. With this, plus LFs for the Magellanic Clouds and a background galaxy luminosity function, and the inputs should be all there. Might have something preliminary working by Thursday.

— Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/76#issuecomment-157461522 .

[yoachim]

Cool. I got my stellar density map working, but it should be upgraded to be a full luminosity function. Anyway, we can hack up some metrics and use a dummy luminosity function(s) and I can fill it in later.

https://github.com/LSST-nonproject/sims_maf_contrib/blob/example-using-maps/tutorials/StellarDensityMapExample.ipynb

[knutago]

Nice! We would probably progress faster if I gave you input and you did hacking, but I will also try making some headway before tomorrow.

[knutago]

Peter, I updated lsst-sims through condo update and switched to your branch, but my version doesn't recognize StellarDensity...does sims-maf need to be installed from source for this to be available?

[yoachim]

I put the stellar density in sims_maf_contrib, so you need that set up as well. And even then, it won't work for you because I haven't packaged up the stellar density maps like the dust maps. I'm at the Lodge now, so if you're around we can try to set you up tonight or first thing tomorrow.

[knutago]

Ok, so here is an iPython notebook which shows a basic version of the CrowdingMetric. Still need to add Magellanic Clouds and background galaxies: https://github.com/knutago/sims_maf_contrib/blob/master/tutorials/CrowdingMetric.ipynb

[willclarkson]

Knut this looks really great!!! We (Milky Way folks) will likely be asking a little more about this in the near-ish future...

Great stuff,

Will

Dr. Will Clarkson Assistant Professor of Physics and Astronomy University of Michigan-Dearborn Students: please put your course code in the title of emails to me (ASTR330 or PHYS 150 lab or discussion).

On Fri, Nov 20, 2015 at 4:38 PM, knutago notifications@github.com wrote:

Ok, so here is an iPython notebook which shows a basic version of the CrowdingMetric. Still need to add Magellanic Clouds and background galaxies: sims_maf_contrib/tutorials/CrowdingMetric.ipynb

— Reply to this email directly or view it on GitHub https://github.com/LSSTScienceCollaborations/ObservingStrategy/issues/76#issuecomment-158531518 .