Closed SimonKrughoff closed 8 years ago
How does opsim report the sky brightness? I can build a new override files for each phoSim run, if necessary.
On 12/10/2015 5:34 PM, SimonKrughoff wrote:
Unfortunately in phosim some observational parameters are actually conflated with the "physics overrides." Examples are cloud opacity and sky brightness. My suggestion for Twinkles 1 has been to turn off clouds so we don't have to worry about that one. Sky brightness, on the other hand, is provided by OpSim. The phosim skybrightness distribution is both too bright and too broad, so I think we should use the OpSim value.
In practice, we'll need a different physics override file for every observation. I'm not sure the best way to do this. Ideally, we could update phosim to read that value from the instance catalog instead of the physics override file, but I'm not sure how feasible that is. We could do this in an ad hoc way for Twinkles 1 and hope for a better solution for Twinkles 2.
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73.
It's trivial in the sense that sky brightness is simply recorded per observation in the OpSim database. It's more a question of how we do it. We currently build in per observation values in the instance catalog (input to phosim) output by catsim, but phosim can't read sky brightness from there. So, really this issue is how do we build the per visit override files?
For reference here are the columns in the table (from OpSim table "Summary"):
Can one simply parse the instance catalog for the sky brightness and then insert the value into the overrides file?
On 12/10/2015 6:00 PM, SimonKrughoff wrote:
It's trivial in the sense that sky brightness is simply recorded per observation in the OpSim database. It's more a question of how we do it. We currently build in per observation values in the instance catalog (input to phosim) output by catsim, but phosim can't read sky brightness from there. So, really this issue is how do we build the per visit override files?
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-163809488.
Here's an example of what's in each ObservationMetaData object returned by opsim:
{'bandpass': 'r',
'boundLength': 0.29999999999999999,
'boundType': 'circle',
'mjd': 49354.074332999997,
'phoSimMetaData': OrderedDict([('Opsim_obshistid', (840, dtype('int64'))),
('SIM_SEED', (92822, dtype('int64'))),
('pointingRA', (0.92518400000000001, dtype('float64'))),
('pointingDec', (-0.47889999999999999, dtype('float64'))),
('Opsim_moonra', (2.6899479999999998, dtype('float64'))),
('Opsim_moondec', (0.104196, dtype('float64'))),
('Opsim_rotskypos', (4.136145, dtype('float64'))),
('Opsim_filter', ('r', dtype('S1'))),
('Opsim_rawseeing', (0.82037700000000002, dtype('float64'))),
('Opsim_sunalt', (-0.37816, dtype('float64'))),
('Opsim_moonalt', (-0.149557, dtype('float64'))),
('Opsim_dist2moon', (1.790632, dtype('float64'))),
('Opsim_moonphase', (80.922484999999995, dtype('float64'))),
('Opsim_expmjd', (49354.074332999997, dtype('float64'))),
('Opsim_altitude', (1.4975560000000001, dtype('float64'))),
('Opsim_azimuth', (5.2548370000000002, dtype('float64'))),
('exptime', (30.0, dtype('float64'))),
('airmass', (1.002688, dtype('float64')))]),
'pointingDec': -27.438948808815123,
'pointingRA': 53.009138473031555,
'rotSkyPos': 236.98365195413788,
'site': <lsst.sims.utils.Site.Site at 0x7f85f92a7110>,
'skyBrightness': 20.859183000000002}
but I don't see skyBrightness being written to the instance catalogs by the .write_catalog(...)
method. We could add that to the instance catalogs, and extract it into a physics commands file for running with phosim, and I'm pretty sure phosim would just ignore it if were an entry it didn't know to process. Alternatively, we could write a physics command file for each visit at the same time we write the instance catalogs. (I'm doing the latter right now using InstcatGenerator.py.)
Producing both an instance catalog and an overrides files at the same time might be the most flexible in the event there may be other quantities passed along that way.
On 12/10/2015 6:22 PM, James Chiang wrote:
Here's an example of what's in each ObservationMetaData object returned by catsim:
|{'bandpass': 'r', 'boundLength': 0.29999999999999999, 'boundType': 'circle', 'mjd': 49354.074332999997, 'phoSimMetaData': OrderedDict([('Opsim_obshistid', (840, dtype('int64'))), ('SIM_SEED', (92822, dtype('int64'))), ('pointingRA', (0.92518400000000001, dtype('float64'))), ('pointingDec', (-0.47889999999999999, dtype('float64'))), ('Opsim_moonra', (2.6899479999999998, dtype('float64'))), ('Opsim_moondec', (0.104196, dtype('float64'))), ('Opsim_rotskypos', (4.136145, dtype('float64'))), ('Opsim_filter', ('r', dtype('S1'))), ('Opsim_rawseeing', (0.82037700000000002, dtype('float64'))), ('Opsim_sunalt', (-0.37816, dtype('float64'))), ('Opsim_moonalt', (-0.149557, dtype('float64'))), ('Opsim_dist2moon', (1.790632, dtype('float64'))), ('Opsim_moonphase', (80.922484999999995, dtype('float64'))), ('Opsim_expmjd', (49354.074332999997, dtype('float64'))), ('Opsim_altitude', (1.4975560000000001, dtype('float64'))), ('Opsim_azimuth', (5.2548370000000002, dtype('float64'))), ('exptime', (30.0, dtype('float64'))), ('airmass', (1.002688, dtype('float64')))]), 'pointingDec': -27.438948808815123, 'pointingRA': 53.009138473031555, 'rotSkyPos': 236.98365195413788, 'site': <lsst.sims.utils.Site.Site at 0x7f85f92a7110>, 'skyBrightness': 20.859183000000002} |
but I don't see skyBrightness being written to the instance catalogs by the |.write_catalog(...)| method. We /could/ add that to the instance catalogs, and extract it into a physics commands file for running with phosim, and I'm pretty sure phosim would just ignore it if were an entry it didn't know to process. Alternatively, we could write a physics command file for each visit at the same time we write the instance catalogs. (I'm doing the latter right now using InstcatGenerator.py https://github.com/DarkEnergyScienceCollaboration/Twinkles/blob/master/twinkles/InstcatGenerator.py.)
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-163816253.
On this part:
Can one simply parse the instance catalog for the sky brightness and then insert the value into the overrides file?
but I don't see skyBrightness being written to the instance catalogs by the .write_catalog(...) method. We could add that to the instance catalogs,
It should be easy to add getters into instance_catalogs or mixins that inherit from instance Catalogs: for example
It is adding the getter for 'band' of the observation metadata, so that it is now available as column of the resulting class. It will still require the 'band' column to be requested at output, for the write_catalog method to write it out.
Trying to draw @danielsf attention, to see if he believes that sky brightness is already in instance catalogs, or should be in instance catalogs, so that it would not need to be added at this stage.
@rbiswas4 it's not in the instance catalog now because phosim doesn't use it. I guess the most "right" way would be to have CatSim generate both the instance catalog and physics override at the same time if it's relatively easy to do.
This should be easy to add to the script generates the instance catalogs from the cadence file. We just need to decide what other overrides should go in that physics commands file.
This isn’t right. Sky brightness cannot get modified by the command file self-consistently. I also disagree with the “too bright and too broad” statement. So please don’t do this.
john
On Dec 10, 2015, at 8:34 PM, SimonKrughoff notifications@github.com wrote:
Unfortunately in phosim some observational parameters are actually conflated with the "physics overrides." Examples are cloud opacity and sky brightness. My suggestion for Twinkles 1 has been to turn off clouds so we don't have to worry about that one. Sky brightness, on the other hand, is provided by OpSim. The phosim skybrightness distribution is both too bright and too broad, so I think we should use the OpSim value.
In practice, we'll need a different physics override file for every observation. I'm not sure the best way to do this. Ideally, we could update phosim to read that value from the instance catalog instead of the physics override file, but I'm not sure how feasible that is. We could do this in an ad hoc way for Twinkles 1 and hope for a better solution for Twinkles 2.
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73.
@johnrpeterson We simply can't use the phosim sky brightness model. I've looked at it several times and it's just not right. For example, back to back exposures can vary by ~2 mags in sky brightness. We've brought this up with the phosim team before, and never gotten anywhere with it.
If phosim can't let us set the sky brightness self-consistently, it's a bug that should be fixed in phosim. Should I file a ticket?
yes, you should certainly file a ticket if that is the case. it should be exactly the same when you do two exposures, so please provide an example of how you did it.
john
On Dec 11, 2015, at 10:59 AM, SimonKrughoff notifications@github.com wrote:
@johnrpeterson https://github.com/johnrpeterson We simply can't use the phosim sky brightness model. I've looked at it several times and it's just not right. For example, back to back exposures can vary by ~2 mags in sky brightness.
If phosim can't let us set the sky brightness self-consistently, that's a bug that should be fixed in phosim. Should I file a ticket?
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-163974102.
@johnrpeterson Turns out we did file a ticket https://jira.lsstcorp.org/browse/PHOSIM-17 . You never commented on it.
this is just simulating a different night every time with the moon completely below the horizon and then you seem to get a ~magnitude of variation from the airglow (which is fine). that’s not really what you said. i thought you said a back to back exposure gets a wide variation.
john
On Dec 11, 2015, at 11:22 AM, SimonKrughoff notifications@github.com wrote:
@johnrpeterson https://github.com/johnrpeterson Turns out we did file a ticket https://jira.lsstcorp.org/browse/PHOSIM-17 https://jira.lsstcorp.org/browse/PHOSIM-17 . You never commented on it.
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-163980003.
@johnrpeterson It's not fine to have an order of magnitude (in flux) variation in dark sky brightness from one adjacent visit to the next! We cannot use that model for anything. It's unphysical and confusing.
We have to be able to set the skybrightness from OpSim.
they aren’t adjacent the way you are simulating them. you are simulating entirely separate nights.
john
On Dec 11, 2015, at 12:02 PM, SimonKrughoff notifications@github.com wrote:
@johnrpeterson https://github.com/johnrpeterson It's not fine to have an order of magnitude (in flux) variation in dark sky brightness from one adjacent visit to the next! We cannot use that model for anything. It's unphysical and confusing.
Conversation moved to the ticket. If we can fix the phosim model, maybe we can use it for Twinkles 2, but until then we need to use OpSim's version of skybrightness.
again, please note that you are simulating entirely different observations that may be separated by days or even years, the way you have done this. if you want to simulate exposures that are closely spaced in time, you have to do it differently.
john
On Dec 11, 2015, at 12:12 PM, SimonKrughoff notifications@github.com wrote:
Conversation moved to the ticket. If we can fix the phosim model, maybe we can use it for Twinkles 2, but until then we need to use OpSim's version of skybrightness.
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-163993573.
@johnrpeterson Can you please move this comment to the ticket, so we can track this problem there instead of on this issue?
Edit: I copied your comment, so we can continue the conversation there.
I copied John's comment over to https://jira.lsstcorp.org/browse/PHOSIM-17, and provided an easy link back to this thread.
On Fri, Dec 11, 2015 at 9:15 AM, SimonKrughoff notifications@github.com wrote:
@johnrpeterson https://github.com/johnrpeterson Can you please move this comment to the ticket, so we can track this problem there instead of on this issue?
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-163994250 .
@drphilmarshall @jchiang87 I've thought about this some more. After looking at the phosim model and at data from Krisciunas et al. (2007), I think the easiest thing to do is to just pick a nominal sky brightness (how about 21.8?). The moon and airmass effects should add significant variance.
The error bars on the V-band sky brightness at zenith are ~+-0.1, so this seems like a reasonable compromise to me.
To outline why I don't think we should try getting the number from OpSim into phosim configuration are:
Selecting a single, nominal sky brightness value sounds fine to me, but wouldn't we still need to set that in a physics commands override file with a zenith_v=21.8
(?) entry? Also, we'd need one anyway to turn off various unmodeled effects. This would enable us to use a single override file for all runs, though.
@jchiang87 That's right. We will need a physics override file, just not one per visit. That will reduce the bookkeeping some.
This is an ok thing to do on a mechanical level, but this seems very arbitrary to me and kind of a snap judgement. Please (Phil?, Michael?) send around a description of the cadence of twinkles and how you think the background subtraction/source detection might affect your science goals. Then i can help make a recommendation based on that.
The reason is that this discussion is too simplistic in the sense that its just about one number. The sky background in phosim is actually a full spatial-spectral-temporal(!) description of the sky in multiple components (moon, zodiacal, airglow, etc.), and the photon propagation of that through clouds/molecular opacity/various forms of instrumental throughput (dust, filters, etc.). So it will vary in time, spatially, filter, airmass… There are known limitations for a particular version of phosim. But needless to say it is important to close the scientific loop here and try to connect the actual science goals to the settings in phosim.
john
On Dec 16, 2015, at 12:52 PM, SimonKrughoff notifications@github.com wrote:
@drphilmarshall https://github.com/drphilmarshall @jchiang87 https://github.com/jchiang87 I've thought about this some more. After looking at the phosim model and at data from Krisciunas et al. (2007), I think the easiest thing to do is to just pick a nominal sky brightness (how about 21.8?). The moon and airmass effects should add significant variance.
The error bars on the V-band sky brightness at zenith are ~+-0.1, so this seems like a reasonable compromise to me.
To outline why I don't think we should try getting the number from OpSim into phosim configuration are:
- It removes the need for a physics command override file.
- Phosim expects the V-band dark sky brightness at zenith and OpSim gives the in-band sky brightness including all effects at the pointing.
- If we are going to go to the trouble of including a non-phosim sky model, I think we should go all the way and integrate a self consistent model like the one Peter Y. has produced.
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165188296.
@johnrpeterson So maybe this is where my confusion comes from. I've asked this on the reopened jira ticket too, what is encoded in the zenith_v number?
I assumed it was the dark sky spectrum normalization and that the moon, zodiacal light, airmass dependence were all imprinted on top of that in a self consistent way, but now it sounds like zenith_v includes (some of) those effects. How exactly does the zenith_v parameter propagate through?
@johnrpeterson We want to be able to report to the DM team on the quality of the extracted light curves from simulated images of one field observed many times.
To do this, we need to be able to generate a consistent set of sequential observations that match the time coherence of the observed sky brightness.
The cadence will be from a recent OpSim run. E.g., Enigma 1189, sky position chosen to be in a deep drilling field.
How should we call PhoSim based on a sequence of observations from OpSim to generate a sky brightness in each image that roughly matches the time correlation of observed sky brightness (captured in this case by OpSim)?
On Dec 16, 2015, at 4:04 PM, Michael Wood-Vasey notifications@github.com wrote:
@johnrpeterson https://github.com/johnrpeterson We want to be able to report to the DM team on the quality of the extracted light curves from simulated images of one field observed many times.
To do this, we need to be able to generate a consistent set of sequential observations that match the time coherence of the observed sky brightness.
The cadence will be from a recent OpSim run. E.g., Enigma 1189, sky position chosen to be in a deep drilling field.
what is the typical time between observations? is this a single field (10 sq. degrees) or multiple fields?
How should we call PhoSim based on a sequence of observations from OpSim to generate a sky brightness in each image that roughly matches the time correlation of observed sky brightness (captured in this case by OpSim)?
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165238786.
@johnrpeterson Can you tell me what is supposed to go into the zenith_v number?
Regarding the typical timing, It will vary because we are planning on simulating both the deep drilling and WFD cadence. Right now, we are only simulating a single chip at the center of one of the DD fields. On nights where the DD cadence is in place, the observations will be back to back with occasional filter changes. On WFD nights, the cadence will be much less frequent: a couple of times a night.
Here is an example of a DD sequence of observations.
time from start (minutes):filter (u=0;g=1;r=2;i=3;z=4;y=5)
0.000 : 2
0.576 : 2
1.296 : 2
1.872 : 2
2.448 : 2
3.024 : 2
3.600 : 2
4.320 : 2
4.896 : 2
5.472 : 2
6.048 : 2
6.624 : 2
7.200 : 2
7.776 : 2
8.496 : 2
9.072 : 2
9.648 : 2
10.224 : 2
10.800 : 2
11.376 : 2
13.968 : 1
14.544 : 1
15.264 : 1
15.840 : 1
16.416 : 1
16.992 : 1
17.568 : 1
18.144 : 1
18.864 : 1
19.440 : 1
22.032 : 3
22.608 : 3
23.184 : 3
23.760 : 3
24.336 : 3
24.912 : 3
25.632 : 3
26.208 : 3
26.784 : 3
27.360 : 3
27.936 : 3
28.512 : 3
29.232 : 3
29.808 : 3
30.384 : 3
30.960 : 3
We're looking at a single deep drilling field in enigma_1189 (fieldID=1427, RA=53.009, Dec=-27.439
). Here is a histogram of the log of the time differences between successive observations, log10(delta t/s)
:
The vertical line is at delta(t)=35 seconds.
Thanks Jim! Other interesting features:
>>> np.log10([34,30*60,24*3600])
array([ 1.53147892, 3.25527251, 4.93651374])
The peak at 3.2 is the transit revisit time, then the one at 4.9 is the "return the following night" peak. Peaks at 5.2 and 5.4 are 2 and 3 days later, respectively.
On Wed, Dec 16, 2015 at 1:59 PM, James Chiang notifications@github.com wrote:
We're looking at a single deep drilling field in enigma_1189 (fieldID=1427, RA=-53.009, Dec=-27.439). Here is a histogram of the log of the time differences between successive observations, log10(delta t/s): [image: delta_t_histogram] https://cloud.githubusercontent.com/assets/1994473/11854646/1d74e8b4-a3fa-11e5-8418-a642c05cc5c4.png The vertical line is at delta(t)=35 seconds.
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165257495 .
Ok very useful, now second question. when, Michael, you say:
"We want to be able to report to the DM team on the quality of the extracted light curves from simulated images of one field observed many times."
what does this mean in detail? there are two ways to go with this. Any scheduler (or virtual scheduler like opsim) will not be able to predict the background perfectly. they will be able to predict the lunar background to some modelling accuracy, but the airglow varies in time and spatially within an lsst field (as well as phosim) so you could either:
1) predict the sky background from a scheduler (i.e. opsim) and then have the background in phosim have some variation compared to what you might predict (which happens in real life)
or
2) use the measured background in phosim to predict how well DM should be able reconstruct the light curves
they are slightly different questions (i.e. a pre-obsevation prediction vs. a post-observation prediction)
john
On Dec 16, 2015, at 5:53 PM, Phil Marshall notifications@github.com wrote:
Thanks Jim! Other interesting features:
>>> np.log10([34,30*60,24*3600]) array([ 1.53147892, 3.25527251, 4.93651374])
The peak at 3.2 is the transit revisit time, then the one at 4.9 is the "return the following night" peak. Peaks at 5.2 and 5.4 are 2 and 3 days later, respectively.
On Wed, Dec 16, 2015 at 1:59 PM, James Chiang notifications@github.com wrote:
We're looking at a single deep drilling field in enigma_1189 (fieldID=1427, RA=-53.009, Dec=-27.439). Here is a histogram of the log of the time differences between successive observations, log10(delta t/s): [image: delta_t_histogram] https://cloud.githubusercontent.com/assets/1994473/11854646/1d74e8b4-a3fa-11e5-8418-a642c05cc5c4.png The vertical line is at delta(t)=35 seconds.
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165257495 .
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165273812.
@johnrpeterson I think we would like to do both. Since we are currently trying to use opsim as our scheduler, can you explain how we would do the first option?
We have a sky model that we are about to release for the LSST. It provides the spatial and temporal variation in the sky brightness validated against the ESO model and CTIO twilight observations. The output is currently a Healpix representation with either an SED or ugrizy surface brightnesses per pixel. What is the best way to integrate this into phosim (thinking of this for Twinkles 2). The model incorporates
cheers Andy
On Thu, Dec 17, 2015 at 1:19 PM, SimonKrughoff notifications@github.com wrote:
@johnrpeterson https://github.com/johnrpeterson I think we would like to do both. Since we are currently trying to use opsim as our scheduler, can you explain how we would do the first option?
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165584534 .
andy-
all that physics is in phosim already. besides being redundant, it wouldn’t be useful for phosim as this isn’t a photon monte carlo of the effects. but it could be useful to compare against and for validation.
regards,
john
On Dec 17, 2015, at 6:26 PM, Andrew Connolly notifications@github.com wrote:
We have a sky model that we are about to release for the LSST. It provides the spatial and temporal variation in the sky brightness validated against the ESO model and CTIO twilight observations. The output is currently a Healpix representation with either an SED or ugrizy surface brightnesses per pixel. What is the best way to integrate this into phosim (thinking of this for Twinkles 2). The model incorporates
- Twilight (sunlight scattered by the atmosphere)
- Zodiacal (sunlight scattered by solar system dust)
- Moon (lunar light scattered by atmosphere)
- Upper and lower atmosphere (line emission)
- Airglow (atmosphere continuum)
- Scattered starlight
- Dark sky contributions
cheers Andy
On Thu, Dec 17, 2015 at 1:19 PM, SimonKrughoff notifications@github.com wrote:
@johnrpeterson https://github.com/johnrpeterson I think we would like to do both. Since we are currently trying to use opsim as our scheduler, can you explain how we would do the first option?
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165584534 .
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165614372.
well basically option #1 and option #2 might involve the same thing, but with option #1 we have to be more careful and make sure we are neither overestimating nor underestimating the expected mismatch between point source sensitivity+photometry errors predicted prior to an observation and what actually occurs. to first order we are already good, because opsim and any scheduler knows to do the bluer bands with the moon is gone and redder bands with the moon is there. so option #2 would lean more towards fixing the airglow to a constant value like we already discussed, but that may underestimate the error so we have to study that. i’d think we should do a mini test of running phosim with opsim parameters and see if the mismatch is believable. option #2 you could run phosim with no special options and just predict how DM should do off the images themselves.
john
On Dec 17, 2015, at 4:20 PM, SimonKrughoff notifications@github.com wrote:
@johnrpeterson https://github.com/johnrpeterson I think we would like to do both. Since we are currently trying to use opsim as our scheduler, can you explain how we would do the first option?
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165584534.
@johnrpeterson Unfortunately, I'm not convinced that the phosim model is good even to first order (the sky background level). I've made a comment on PHOSIM-17 and I think I've shown that phosim overestimates the measured variation in the dark sky by a factor of many (in magnitude!) compared to data taken at CTIO. My hope is that I'm just misusing phosim somehow. I'm open to suggestions.
Until we can do the minimum level of validation, I think we have to go with the second option. We'll have to do more detailed validation, but I don't think it's worth our time until we can show that phosim can reproduce the broad statistical properties of the sky background.
Given all of this conversation I think the simplest course of action would be to compare the outputs of the models from phosim with the new LSST model (not the current opsim as that is a fairly adhoc implementation of Krisciunas and Schaefer). I think that is consistent with @johnrpeterson https://github.com/johnrpeterson suggestion for some mini runs and would be a good validation. Is it possible for phosim to be run in a mode (or modified to run in a mode) where it will output the sky brightness for a given ra,dec (given the moon, sun configuration) rather than generate the full image?
cheers Andy
On Fri, Dec 18, 2015 at 8:37 AM, SimonKrughoff notifications@github.com wrote:
@johnrpeterson https://github.com/johnrpeterson Unfortunately, I'm not convinced that the phosim model is good even to first order (the sky background level). I've made a comment on PHOSIM-17 and I think I've shown that phosim overestimates the measured variation in the dark sky by a factor of many (in magnitude!) compared to data taken at CTIO. My hope is that I'm just misusing phosim somehow. I'm open to suggestions.
Until we can do the minimum level of validation, I think we have to go with the second option. We'll have to do more detailed validation, but I don't think it's worth our time until we can show that phosim can reproduce the broad statistical properties of the sky background.
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165828903 .
i thought of one potential problem though that could prevent the loop on this issue from ever closing. basically there is a design flaw on this issue in opsim in that the sky brightness should not be perfectly known to the scheduler. the actual observational data from auxillary instruments (i.e. phosim) should be the input to the scheduler (i.e. opsim) and also opsim can have its own internal modelling (which should be independent), but not the other way around. otherwise, its impossible to guess how well the scheduler can optimize the survey to minimize sky background. in this way, opsim is assuming perfect knowledge, which is impossible given the considerable time variation of the sky. in other words, there should a mismatch at some level between the actual phosim intensity and what opsim thinks it should be, but it is hard to know how to predict this mismatch with the way opsim works. but this is not easy to correct, i’d suspect.
i’ll have to think about this more over the holidays.
more later,
john
On Dec 18, 2015, at 1:08 PM, Andrew Connolly notifications@github.com wrote:
Given all of this conversation I think the simplest course of action would be to compare the outputs of the models from phosim with the new LSST model (not the current opsim as that is a fairly adhoc implementation of Krisciunas and Schaefer). I think that is consistent with @johnrpeterson https://github.com/johnrpeterson suggestion for some mini runs and would be a good validation. Is it possible for phosim to be run in a mode (or modified to run in a mode) where it will output the sky brightness for a given ra,dec (given the moon, sun configuration) rather than generate the full image?
cheers Andy
On Fri, Dec 18, 2015 at 8:37 AM, SimonKrughoff notifications@github.com wrote:
@johnrpeterson https://github.com/johnrpeterson Unfortunately, I'm not convinced that the phosim model is good even to first order (the sky background level). I've made a comment on PHOSIM-17 and I think I've shown that phosim overestimates the measured variation in the dark sky by a factor of many (in magnitude!) compared to data taken at CTIO. My hope is that I'm just misusing phosim somehow. I'm open to suggestions.
Until we can do the minimum level of validation, I think we have to go with the second option. We'll have to do more detailed validation, but I don't think it's worth our time until we can show that phosim can reproduce the broad statistical properties of the sky background.
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165828903 .
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165859211.
I spent some time looking at the sky brightnesses in phosim and the predictions from the new LSST sky model (derived from the ESO model) and ran into a discrepancy that it would be good to get some input on. I simulated a couple of nights (one with a rising full moon and another with a rising 50% moon) with a pointing at the zenith and a set of 15 second exposures spread out through the night (all using the same seed as input to phosim). I use a simulated 20th magnitude source to define the zeropt for the phosim images (the zero point appears to be about 20% different from the current nominal values used by the project) and use this to calibrate the sky brightness. The results are attached. I think I must be an error somewhere in my phosim configuration files (e.g. I switch off airglow variation in the override file and remove clouds) as the phosim skies are about 2-3 magnitudes brighter than the ESO model once the moon comes up (e.g. in g band the sky brightens 6 mags with phosim but 3 mags with the ESO model). I attached one of input files I am using in case there is an obvious error in the config. Could someone take a quick look.
Couple of points on the simulation - simulating a bright sky in phosim takes about 30 hrs for a 100x100 patch. It would be great if we could have a method that would output an estimate of the sky without having to generate the full image as this is the limiting factor on this comparison.
cheers Andy
On Tue, Dec 22, 2015 at 9:44 AM, johnrpeterson notifications@github.com wrote:
i thought of one potential problem though that could prevent the loop on this issue from ever closing. basically there is a design flaw on this issue in opsim in that the sky brightness should not be perfectly known to the scheduler. the actual observational data from auxillary instruments (i.e. phosim) should be the input to the scheduler (i.e. opsim) and also opsim can have its own internal modelling (which should be independent), but not the other way around. otherwise, its impossible to guess how well the scheduler can optimize the survey to minimize sky background. in this way, opsim is assuming perfect knowledge, which is impossible given the considerable time variation of the sky. in other words, there should a mismatch at some level between the actual phosim intensity and what opsim thinks it should be, but it is hard to know how to predict this mismatch with the way opsim works. but this is not easy to correct, i’d suspect.
i’ll have to think about this more over the holidays.
more later,
john
On Dec 18, 2015, at 1:08 PM, Andrew Connolly notifications@github.com wrote:
Given all of this conversation I think the simplest course of action would be to compare the outputs of the models from phosim with the new LSST model (not the current opsim as that is a fairly adhoc implementation of Krisciunas and Schaefer). I think that is consistent with @johnrpeterson https://github.com/johnrpeterson suggestion for some mini runs and would be a good validation. Is it possible for phosim to be run in a mode (or modified to run in a mode) where it will output the sky brightness for a given ra,dec (given the moon, sun configuration) rather than generate the full image?
cheers Andy
On Fri, Dec 18, 2015 at 8:37 AM, SimonKrughoff <notifications@github.com
wrote:
@johnrpeterson https://github.com/johnrpeterson Unfortunately, I'm not convinced that the phosim model is good even to first order (the sky background level). I've made a comment on PHOSIM-17 and I think I've shown that phosim overestimates the measured variation in the dark sky by a factor of many (in magnitude!) compared to data taken at CTIO. My hope is that I'm just misusing phosim somehow. I'm open to suggestions.
Until we can do the minimum level of validation, I think we have to go with the second option. We'll have to do more detailed validation, but I don't think it's worth our time until we can show that phosim can reproduce the broad statistical properties of the sky background.
— Reply to this email directly or view it on GitHub < https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165828903
.
— Reply to this email directly or view it on GitHub < https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165859211 .
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-166686824 .
Andy, we are validating against the real sky measurements now as this will be the only way to check the short-term variability. as for what you said below, if you send me the set of catalogs you used we can look into this futher.
john
On Jan 7, 2016, at 10:43 PM, Andrew Connolly notifications@github.com wrote:
I spent some time looking at the sky brightnesses in phosim and the predictions from the new LSST sky model (derived from the ESO model) and ran into a discrepancy that it would be good to get some input on. I simulated a couple of nights (one with a rising full moon and another with a rising 50% moon) with a pointing at the zenith and a set of 15 second exposures spread out through the night (all using the same seed as input to phosim). I use a simulated 20th magnitude source to define the zeropt for the phosim images (the zero point appears to be about 20% different from the current nominal values used by the project) and use this to calibrate the sky brightness. The results are attached. I think I must be an error somewhere in my phosim configuration files (e.g. I switch off airglow variation in the override file and remove clouds) as the phosim skies are about 2-3 magnitudes brighter than the ESO model once the moon comes up (e.g. in g band the sky brightens 6 mags with phosim but 3 mags with the ESO model). I attached one of input files I am using in case there is an obvious error in the config. Could someone take a quick look.
Couple of points on the simulation - simulating a bright sky in phosim takes about 30 hrs for a 100x100 patch. It would be great if we could have a method that would output an estimate of the sky without having to generate the full image as this is the limiting factor on this comparison.
cheers Andy
On Tue, Dec 22, 2015 at 9:44 AM, johnrpeterson notifications@github.com wrote:
i thought of one potential problem though that could prevent the loop on this issue from ever closing. basically there is a design flaw on this issue in opsim in that the sky brightness should not be perfectly known to the scheduler. the actual observational data from auxillary instruments (i.e. phosim) should be the input to the scheduler (i.e. opsim) and also opsim can have its own internal modelling (which should be independent), but not the other way around. otherwise, its impossible to guess how well the scheduler can optimize the survey to minimize sky background. in this way, opsim is assuming perfect knowledge, which is impossible given the considerable time variation of the sky. in other words, there should a mismatch at some level between the actual phosim intensity and what opsim thinks it should be, but it is hard to know how to predict this mismatch with the way opsim works. but this is not easy to correct, i’d suspect.
i’ll have to think about this more over the holidays.
more later,
john
On Dec 18, 2015, at 1:08 PM, Andrew Connolly notifications@github.com wrote:
Given all of this conversation I think the simplest course of action would be to compare the outputs of the models from phosim with the new LSST model (not the current opsim as that is a fairly adhoc implementation of Krisciunas and Schaefer). I think that is consistent with @johnrpeterson https://github.com/johnrpeterson suggestion for some mini runs and would be a good validation. Is it possible for phosim to be run in a mode (or modified to run in a mode) where it will output the sky brightness for a given ra,dec (given the moon, sun configuration) rather than generate the full image?
cheers Andy
On Fri, Dec 18, 2015 at 8:37 AM, SimonKrughoff <notifications@github.com
wrote:
@johnrpeterson https://github.com/johnrpeterson Unfortunately, I'm not convinced that the phosim model is good even to first order (the sky background level). I've made a comment on PHOSIM-17 and I think I've shown that phosim overestimates the measured variation in the dark sky by a factor of many (in magnitude!) compared to data taken at CTIO. My hope is that I'm just misusing phosim somehow. I'm open to suggestions.
Until we can do the minimum level of validation, I think we have to go with the second option. We'll have to do more detailed validation, but I don't think it's worth our time until we can show that phosim can reproduce the broad statistical properties of the sky background.
— Reply to this email directly or view it on GitHub < https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165828903
.
— Reply to this email directly or view it on GitHub < https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-165859211 .
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-166686824 .
— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/73#issuecomment-169884578.
@johnrpeterson Catalog is attached to the previous message in the thread (the one that isnt a png image). Can you let me know if there is anything obviously wrong and that you get the same counts (20K electrons per pixel for this configuration). Thanks
@connolly I'd like to close this since we've come to consensus on the physics override file for Twinkles I. Are you tracking the moon brightness issue elsewhere, and if so, can you link it here so we have a record?
Unfortunately in phosim some observational parameters are actually conflated with the "physics overrides." Examples are cloud opacity and sky brightness. My suggestion for Twinkles 1 has been to turn off clouds so we don't have to worry about that one. Sky brightness, on the other hand, is provided by OpSim. The phosim skybrightness distribution is both too bright and too broad, so I think we should use the OpSim value.
In practice, we'll need a different physics override file for every observation. I'm not sure the best way to do this. Ideally, we could update phosim to read that value from the instance catalog instead of the physics override file, but I'm not sure how feasible that is. We could do this in an ad hoc way for Twinkles 1 and hope for a better solution for Twinkles 2.