Sky treatment does not account atmospheric extinction

[gdhungana]

Currently in quicksim, the sky is treated from top of the atmosphere, not accounting the extinction. I think (from one of the telecon) the zeroth order convention was to put atmospheric extinction to not only the source but sky also.

[dkirkby]

I agree. I have already updated the user's guide to reflect this change, but I have not yet updated the code, which still matches the original IDL version in this respect.

[dkirkby]

I was hoping this would be as simple as copying & pasting one line of code, but it will a bit more complicated because I am currently pre-computing the resolution-convolved sky in each camera as part of the initialization phase, but now that will need to be moved into the per-spectrum simulation code since the extinction depends on the per-observation airmass.

[julienguy]

"I think (from one of the telecon) the zeroth order convention was to put atmospheric extinction to not only the source but sky also. " I am not sure it is a good idea. The sky flux model we use are from observations for which no atmospheric absorption correction has been applied if I am not mistaken. In any case we don't know what absorption to apply, it is certainly different than that of stars. Introducing an atmospheric correction to the sky is likely to introduce some misunderstandings later.

[dkirkby]

@julienguy I had similar concerns and I don't recall now why we decided to make this change. Perhaps @sbailey remembers?

Any model that treats atmospheric emission and extinction as independent processes is going to have problems, but we might as well adopt the "least wrong" approach, then make sure that our inputs are consistent with this approach (changing them if necessary) and, finally, try to estimate the science impact of not using a more realistic model.

[sbailey]

Notes from Rich Kron sent to desi-data:

Notes on sky spectra R. Kron 27 August 2015

A primary source for the (dark) sky spectrum is that from Paranal, the "UVES" spectrum (R.W. Hanuschik 2003):

http://www.aanda.org/articles/aa/abs/2003/33/aa3874/aa3874.html

This has high S/N and R ~ 43,000 and covers the whole spectral range. It does not show us how to predict the dependence of the sky spectrum on zenith angle, but this issue is discussed in Section 4.2 of the paper.

The sky at Kitt Peak has also been measured at high resolution, albeit in a venerable (= data not digital) paper by A. L. Broadfoot and K. R. Kendall 1968:

http://onlinelibrary.wiley.com/doi/10.1029/JA073i001p00426/abstract

Phil Massey and colleagues have been tracking how the night sky spectrum at Kitt Peak varies with time and with direction; the most recent paper is Neugent, K. and Massey, P. 2010:

http://adsabs.harvard.edu/abs/2010PASP..122.1246N

These last spectra only have R ~ 750 and range 3700 - 6800 A, but they do illustrate the features due to light pollution as seen from Kitt Peak.

To the extent we want more information on light pollution, the Lick sky spectrum (3800 - 9200 A) at high resolution is discussed by T.G. Slanger et al. 2003:

http://adsabs.harvard.edu/abs/2003PASP..115..869S

A cursory look through the paper did not reveal where to get the actual spectrum (as opposed to sections of it), but I suspect we can get a file from the authors.

The telluric absorption spectrum is also important - it is expected to be present in front of the upper-atmosphere emission, but will be mixed in with the light scattered back from terrestrial sources at lower altitude. The UVES paper talks about the telluric spectrum at Paranal. For Kitt Peak, there is a new reference (extracted from the solar spectrum 2958 - 9250 A) by Wallace, Hinkle, Livingston, Davis 2011:

http://iopscience.iop.org/0067-0049/195/1/6/article

but again we may need to ask to get a file.

Perhaps the next step is to ask whether using the UVES spectrum leads to any significant shortcomings in DESI simulations because it does not contain features from light pollution. It could be that the addition of moonlight is much more important, but still maybe worth thinking about.

[sbailey]

This first reference about the UVES spectra that we use includes the following discussion about extinction corrections for the spectra. They recognize that the model isn't perfect, but they argue that the primary effect is emission happening above absorption, and they attempt to correct for that absorption.

Extinction correction. The major part of the sky emission spectrum is due to OH Meinel bands and arises in atmospheric heights of typically 90 km (Leinert et al. 1998). Light emitted at those altitudes can be expected to suffer from essentially the same extinction as light from extraterrestrial sources. However, the emission layer of finite distance and extent introduces a spe- cial geometric effect. The optical path length through a layer of thickness l0 along a line of sight at airmass X increases with X, l ≈ l0 · X (Roach & Meinel 1955; Garstang 1989). This causes the night sky emission to brighten towards the horizon.

This is applicable, however, for only a fraction f of the con- tinuum sky emission which is of atmospheric origin, while the remaining fraction (1− f ) is from diffuse extraterrestrial sources and does not show that effect (Patat 2003).

Assumptions on f are very uncertain. It is also well known that sky emission is highly variable (see Patat 2003). Therefore only a standard extinction correction was applied which fo- cuses on the leading effect, namely the extinction which is strongly chromatic, especially in the blue:

E(λ, X) = 10−0.4 e(λ)X .

The standard La Silla extinction law e, extrapolated beyond 9000 Å, has been used (Tu ̈g 1980) which takes into account Rayleigh scattering, ozone and aerosol absorption. The mean airmass values from Table 1 have been used.

Neglecting the geometric brightening in principle leads to an overcorrection of extinction, and hence the sky emission flux. Using the UBVRI extinction coefficients for Paranal from Patat’s Table A.1, the mean airmass values from Table 1, and f = 0.6 (Patat’s Appendix C), the effect of the overshoot can be estimated. The result is that on average the corrected fluxes may be too high by about 0.10 mag (less in the red, more in the 437 setting with its higher average airmass), with no strong wavelength dependence. We will ignore that potential effect in the following.

[dkirkby]

The recently tagged v0.3 includes a configuration option to apply extinction to the sky emission:

    # Should extinction be applied to the sky emission?
    extinct_emission: no

This option is currently turned off in desi.yaml, for backwards compatibility, but we can now fix this issue by turning it on. Do we still want to do this? Are there any tests / plots you want to see first?

[dkirkby]

Hearing no complaints, I have now flipped the switch in PR #33.

For a 1000s exposure of an AB-22 source at airmass 1.0, the median SNR changes from 0.9338 per 0.5A pixel to 0.9687, i.e., an increase in SNR of about 4%.