Implement realistic sky model

[cwwalter]

It would be good to use the work from @yoachim and @dkirkby.

[yoachim]

https://github.com/lsst/sims_skybrightness

[cwwalter]

Thanks!

Can this be easily used as is to add the proper sky model in our program?

Also, have you thought about coordinating with @dkirkby to incorporate the material he showed about the sky at the site?

[yoachim]

It should be good to go--give it an MJD and sky coordinates and it can return the spectrum of the background sky.

I'm not currently planning on adding more to the sky model unless there's a strong demand for it.

[cwwalter]

It should be good to go--give it an MJD and sky coordinates and it can return the spectrum of the background sky.

Great! Will also give the correct spatial variation?

I'm not currently planning on adding more to the sky model unless there's a strong demand for it.

I thought that @dkirkby showed in a SSim meeting there was at least one large spectral feature missing from the base ESO model.

[yoachim]

There is spatial variation in there--if you go closer to the moon it gets brighter. That's sampled on a fairly rough scale, so if you're looking for variation across a single chip the sky brightness code just interpolates from the low res grid. But yes, you can give it multiple ra,dec pointings and get back multiple spectra that should be varying in the correct way.

I think most of us suspect that feature has to be from light pollution at APO? Which may or may not be an issue in Chile? We used the all-sky camera on Pachon to check the sky model and got fairly good agreement, so if there are missing spectral features, they don't seem to be skewing the final broad band magnitudes much. I did try to make the code as modular as possible, so it should be easy to drop in a new spectral component if we want to.

[cwwalter]

There is spatial variation in there--if you go closer to the moon it gets brighter. That's sampled on a fairly rough scale, so if you're looking for variation across a single chip the sky brightness code just interpolates from the low res grid. But yes, you can give it multiple ra,dec pointings and get back multiple spectra that should be varying in the correct way.

Is the moon position an input to the routine?

[yoachim]

If you use the setRADecMjd method, it assumes you're at the LSST site and computes moon and sun position for you. There is also an under-documented setParams method where you can set all the relevant parameters manually if you really want to. There's no sanity checking with setParams, so you can put a full moon at zenith at sunset if you want to.

[cwwalter]

@jchiang87 and @sethdigel: In the phosim files the moon position is set correct? Or is that just an override if you don't want do something like Peter just described?

[dkirkby]

The ESO model does not reproduce details of actual BOSS sky spectra, but its not clear how much that matters for LSST simulations. Here are some plots I showed at the June SSIM meeting:

Typical BOSS sky: (flux vs wavelength in Angstroms)

ESO model matched to this BOSS spectrum:

The most obvious discrepancy is the bump around 6000A, which is also seen at Cerro Paranal (so not due to APO light pollution) and attributed to airglow in this paper (see Fig. 16).

[cwwalter]

I'm not currently planning on adding more to the sky model unless there's a strong demand for it.

Based on @dkirkby's work, it sounds like the airglow feature at 6000Å might be relevant/important.

[cwwalter]

It should be good to go--give it an MJD and sky coordinates and it can return the spectrum of the background sky.

So if we want this to vary over the chip, we should call it at every pixel position?

• Do we want that? Or do we want to set a background per chip?
• @danielsf Is varying over the chip something that can be added relatively naturally in a sky model here:

https://github.com/lsst/sims_GalSimInterface/blob/master/python/lsst/sims/GalSimInterface/galSimNoiseAndBackground.py#L102

It looks to me like this might give you the same background over the entire focal plane now. Is that right?

[danielsf]

Yes, the current implementation gives you the same background over the whole chip, but it should be easy to write a different class that varies the background over the chip and just stick it in where the current class is used. You would just need to write a sub-class of NoiseAndBackgroundBase with a addNoiseAndBackground() method that does whatever you want it to do.

[cwwalter]

Yes, the current implementation gives you the same background over the whole chip,

Is it per chip, or per focal plane? I don't see how you add it differently for different sensors..

[danielsf]

Right now it is per focal plane, but the addNoiseAndBackground method accepts a FITS image (i.e. a detector) which it then acts on, so it wouldn't take much work to add another parameter telling the method which chip this actually is and then have the internal logic do something appropriate.

[cwwalter]

Some initial code for this is now in Pull Request #40.

[cwwalter]

I have been trying to simulate a single star (at the center of the field) using the same instance file in imSim and PhoSim to compare the background levels as @dkirkby did in:

https://github.com/LSSTDESC/imSim/pull/4#issuecomment-254674885

I am using the branch I created in #40 and PhoSim 3.6 (latest release version). In addition, I modified obs_lsstSim to not apply chip position offsets and applied the 'clearpertubations' command in PhoSim (which results in an ideal design removing all instrumental perturbations). The effect of this should be to make the resulting fits files directly comparable. I am using a single 15 second exposure.

The instance file (which was generated for me at some point by @danielsf) is as follows:

rightascension 31.1133844
declination -10.0970060
mjd 59797.2854090
altitude 43.6990272
azimuth 73.7707957
filter 2
rotskypos 69.0922930
rottelpos 0.000
dist2moon 145.1095257
moonalt -11.1383568
moondec -18.7702120
moonphase 59.6288830
moonra 230.9832941
nsnap 1
obshistid 161899
seed 161899
seeing 0.7613760
sunalt -59.1098785
vistime 15.0000000
object 1046817878020 31.1133844 -10.0970060 22.0 starSED/phoSimMLT/lte033-4.5-1.0a+0.4.BT-Settl.spec.gz 0 0 0 0 0 0 point none CCM 0.0635117705 3.1

When I run this through imSim I find a background sky level of about 545 electrons/pixel. However, when I run this same file through PhoSim I get a background level about 1/10 as large (mean level is 65 electrons/pixel). The example/star file supplied with PhoSim results in a background level of about 450 which is closer.

To track down what was going on, I started with the examples/star file and line by line began to modify it, making it more and more like the imSim file until suddenly it started to report an anomalously low background level. Of course, it was the very last line I changed in the file that suddenly made the difference.

This effect happens when the Altitude line in:

rightascension 31.1133844
declination -10.0970060
mjd 59797.2854090
altitude 43.6990272
azimuth 73.7707957

is changed. In the default example/star file the equivalent values are:

altitude 89
azimuth 0

If I change the altitude line (not changing the RA and DEC values) the value changes dramatically.

Here is how the electrons/pixel level change as a function of the altitude parameter (changing the azimuth has no effect):

Altitude 5 degrees: 36
Altitude 10 degrees: 49
Altitude 25 degrees: 60
Altitude 45 degrees: 65
Altitude 65 degrees: 102
Altitude 89 degrees: 351
Altitude 160 degrees: 58

So, at the default value of 90 degrees we get something reasonable. Do any twinkles people have insight into the order or precedence of the pointing commands? You are allowed to give inconsistent values between these two sets of pointing commands in PhoSim. I think the numbers from Scott are all consistent. In imSim only the RA and DEC are considered.

Also, fundamentally, I can't understand why the sky gets dimmer as we go towards the horizon. The only thing that comes to mind is we are getting closer to the moon when pointed straight up but according to the file, it should be below the horizon. Does anyone have any ideas?

At the maximum sky brightness with an altitude of 89 degrees the comparison with the imSim file (with only that line changed) looks like this:

So, it seems to me there are two issues to consider here:

• What is the "correct" sky level?
• Is this comparison with PhoSim correct/good enough to draw conclusions, or is there either a bug in the way I am using the program or the program itself.

Any comments would be welcome as we need to fix this to go ahead with our production.

[dkirkby]

An altitude of 160deg should raise an exception, but perhaps that's a clue. Is this really azimuth angle (i.e., measured away from vertical) and not altitude angle (measured above the horizon) ?

[tonytyson]

Perhaps alt = 90 - correct.alt

Sent from my iPhone

On Jan 6, 2017, at 10:57, Chris Walter notifications@github.com wrote:

I have been trying to simulate a single star (at the center of the field) using the same instance file in imSim and PhoSim to compare the background levels as @dkirkby did in:

4 (comment)

I am using the branch I created in #40 and PhoSim 3.6 (latest release version). In addition, I modified obs_lsstSim to not apply chip position offsets and applied the 'clearpertubations' command in PhoSim (which results in an ideal design removing all instrumental perturbations). The effect of this should be to make the resulting fits files directly comparable. I am using a single 15 second exposure.

The instance file (which was generated for me at some point by @danielsf) is as follows:

rightascension 31.1133844 declination -10.0970060 mjd 59797.2854090 altitude 43.6990272 azimuth 73.7707957 filter 2 rotskypos 69.0922930 rottelpos 0.000 dist2moon 145.1095257 moonalt -11.1383568 moondec -18.7702120 moonphase 59.6288830 moonra 230.9832941 nsnap 1 obshistid 161899 seed 161899 seeing 0.7613760 sunalt -59.1098785 vistime 15.0000000 object 1046817878020 31.1133844 -10.0970060 22.0 starSED/phoSimMLT/lte033-4.5-1.0a+0.4.BT-Settl.spec.gz 0 0 0 0 0 0 point none CCM 0.0635117705 3.1 When I run this through imSim I find a background sky level of about 545 electrons/pixel. However, when I run this same file through PhoSim I get a background level about 1/10 as large (mean level is 65 electrons/pixel). The example/star file supplied with PhoSim results in a background level of about 450 which is closer.

To track down what was going on, I started with the examples/star file and line by line began to modify it, making it more and more like the imSim file until suddenly it started to report an anomalously low background level. Of course, it was the very last line I changed in the file that suddenly made the difference.

This effect happens when the Altitude line in:

rightascension 31.1133844 declination -10.0970060 mjd 59797.2854090 altitude 43.6990272 azimuth 73.7707957 is changed. In the default example/star file the equivalent values are:

altitude 89 azimuth 0 If I change the altitude line (not changing the RA and DEC values) the value changes dramatically.

Here is how the electrons/pixel level change as a function of the altitude parameter (changing the azimuth has no effect):

Altitude 5 degrees: 36 Altitude 10 degrees: 49 Altitude 25 degrees: 60 Altitude 45 degrees: 65 Altitude 65 degrees: 102 Altitude 89 degrees: 351 Altitude 160 degrees: 58 So, at the default value of 90 degrees we get something reasonable. Do any twinkles people have insight into the order or precedence of the pointing commands? You are allowed to give inconsistent values between these two sets of pointing commands in PhoSim. I think the numbers from Scott are all consistent. In imSim only the RA and DEC are considered.

Also, fundamentally, I can't understand why the sky gets dimmer as we go towards the horizon. The only thing that comes to mind is we are getting closer to the moon when pointed straight up but according to the file, it should be below the horizon. Does anyone have any ideas?

At the maximum sky brightness with an altitude of 89 degrees the comparison with the imSim file (with only that line changed) looks like this:

So, it seems to me there are two issues to consider here:

What is the "correct" sky level? Is this comparison with PhoSim correct/good enough to draw conclusions, or is there either a bug in the way I am using the program or the program itself. Any comments would be welcome as we need to fix this to go ahead with our production.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[SimonKrughoff]

I don't believe that phosim does any internal checking for consistency of input parameters.

[cwwalter]

For this work, I think we should try to compare the results of this code against some other reference standard aside from phosim, especially given that that we don't understand what phosim is doing (or so it seems).

Do you have a suggestion for what to compare against?

I'm afraid not, though perhaps Peter or David can provide some suggestions for testing this.

@dkirkby and @yoachim I'm copying this in from an email thread between @jchiang87 and myself. Do either of you have a suggestion for validating this?

[cwwalter]

OK I updated to @fjaviersanchez's new zeropoint numbers:

zp_u =  0.282598538804 1 / (m2 s)
zp_g =  1.56492206349 1 / (m2 s)
zp_r =  1.3488785992 1 / (m2 s)
zp_i =  0.998783620839 1 / (m2 s)
zp_z =  0.699897205858 1 / (m2 s)
zp_y =  0.32665889564 1 / (m2 s)

@jchiang87 Actually implemented a config section where we can specify them.

After making this change the GalSim sky level changed from about 535 to about 438 electrons/pixel/15 seconds. The agreement with PhoSim (at 89 degrees) is now somewhat closer (a factor of 1.3 instead of 1.6):

[cwwalter]

Note: pull request #40 was implemented. Now the master uses @yoachim's ESO model for setting the sky level. However, we have the outstanding issue of verifying that the sky level is correct and currently the sky level is constant over the entire field. It is set by the value given at the pointing.

So, good progress but not yet complete so this issue will stay open.

[fjaviersanchez]

@cwwalter Those zeropoint values are at airmass 1.2. They include aerosols, atmosphere (median values for the site), the latest engineering model for the lenses, the mirror, the detector, and the filters. Do you have any other component (or maybe different conditions?) that can change the throughput values?

[cwwalter]

@fjaviersanchez

Those zeropoint values are at airmass 1.2. They include aerosols, atmosphere (median values for the site), the latest engineering model for the lenses, the mirror, the detector, and the filters. Do you have any other component (or maybe different conditions?) that can change the throughput values?

Do you mean in the PhoSim setup?

[fjaviersanchez]

Yes, like if you include any other component in the PhoSim setup to decrease the signal.

[cwwalter]

It is using the default PhoSim model. There are aerosols, sky glow etc in it but I think that should be in Peter's model too.

[cwwalter]

I have put remaining outstanding issues related to the sky model into their own issues:

45 - Use system photUtils to calculate the zeropoints

46 - Make the model vary over the focal plane

47 - Finalize the level comparison by making sure we understand the discrepancies with PhoSim.

I am closing this issue as the most basic functionality is now implemented.