PhoSim "smoke test"

[drphilmarshall]

Let's add an option to the script that calls PhoSim to make it carry out a smoke test: edit the instance catalog to turn down the exposure time to 0.1s (or maybe less) and run the job, hopefully in a few minutes. Output directory can be treated as scratch space where files can be happily over-written.

[drphilmarshall]

Turn both times (_exposure and _visit) to 0.1 until we find out which one is being used....

[jchiang87]

Based on this code (see also here and here), exptime is the entry that is being used to set the number of photons simulated for each source. It's read in as a double, so we can make it very small. It looks like SIM_VISTIME isn't used.

[johnrpeterson]

its SIM_VISTIME that would get used. that gets used to calculate the exposure time based on the number of snaps and the readout time.

On Feb 25, 2016, at 8:07 PM, James Chiang notifications@github.com<mailto:notifications@github.com> wrote:

Based on this codehttps://bitbucket.org/phosim/phosim_release/src/d912a460f8783c176146d4cdd4149337d3d5f822/source/raytrace/observation.cpp?at=master&fileviewer=file-view-default#observation.cpp-1140 (see also herehttps://bitbucket.org/phosim/phosim_release/src/d912a460f8783c176146d4cdd4149337d3d5f822/source/raytrace/telescopesetup.cpp?at=master&fileviewer=file-view-default#telescopesetup.cpp-585 and herehttps://bitbucket.org/phosim/phosim_release/src/d912a460f8783c176146d4cdd4149337d3d5f822/source/raytrace/photonloop.cpp?at=master&fileviewer=file-view-default#photonloop.cpp-85), exptime is the entry that is being used to set the number of photons simulated for each source. It's read in as a double, so we can make it very small. It looks like SIM_VISTIME isn't used.

— Reply to this email directly or view it on GitHubhttps://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/144#issuecomment-189062247.

[drphilmarshall]

Aha! John: if we wanted to simulate a visit image with 0.01 seconds exposure time, would we need to set both SIM_VISTIME and exptime to 0.01? What happens if we choose these times to be different? Any other advice about designing this sort of PhoSim dry run would be much appreciated!

On Fri, Feb 26, 2016 at 6:29 AM, johnrpeterson notifications@github.com wrote:

its SIM_VISTIME that would get used. that gets used to calculate the exposure time based on the number of snaps and the readout time.

On Feb 25, 2016, at 8:07 PM, James Chiang <notifications@github.com mailto:notifications@github.com> wrote:

Based on this code< https://bitbucket.org/phosim/phosim_release/src/d912a460f8783c176146d4cdd4149337d3d5f822/source/raytrace/observation.cpp?at=master&fileviewer=file-view-default#observation.cpp-1140> (see also here< https://bitbucket.org/phosim/phosim_release/src/d912a460f8783c176146d4cdd4149337d3d5f822/source/raytrace/telescopesetup.cpp?at=master&fileviewer=file-view-default#telescopesetup.cpp-585> and here< https://bitbucket.org/phosim/phosim_release/src/d912a460f8783c176146d4cdd4149337d3d5f822/source/raytrace/photonloop.cpp?at=master&fileviewer=file-view-default#photonloop.cpp-85>), exptime is the entry that is being used to set the number of photons simulated for each source. It's read in as a double, so we can make it very small. It looks like SIM_VISTIME isn't used.

— Reply to this email directly or view it on GitHub< https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/144#issuecomment-189062247>.

— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/144#issuecomment-189303606 .

[SimonKrughoff]

@drphilmarshall @jchiang87 I actually thought the exptime input variable was ignored and that the SIM_* inputs were the important ones. I'll try to find time to look more tomorrow.

[jchiang87]

Yes, I think you and John are right about that. When I grepped the raytrace code I didn't notice that even though exptime is read in from the .pars file, it is recalculated here based on the value of vistime, which in turn is set to be the same as SIM_VISTIME by phosim.py.

[SimonKrughoff]

O.K. I missed John's comment. It's fairly difficult trace the path of input variables through the phosim code. It would be nice if that were easier.

[jchiang87]

Looking at that code I linked to more carefully, and re-reading John's comment, I see that we need to consider the readout times for the two types of devices in focalplanelayout.txt. It looks like we can't pick a SIM_VISTIME that is shorter than the readout time for the CCD devices (as opposed to CMOS), which is 3.0 sec in data/lsst/focalplanelayout.txt, otherwise exptime would become negative, and a negative value would crash the number of photon Poisson draw for each source.

[johnrpeterson]

No that’s not quite right. you can pick a small exposure time.

for the science CCD’s its:

visit time = exposure time * number of snaps + readout time * (number of snaps - 1)

for the guiders its different.

we recently update the user documentation on the wiki to describe all of this, so you can find that there.

john

On Feb 29, 2016, at 2:12 AM, James Chiang notifications@github.com<mailto:notifications@github.com> wrote:

Looking at that code I linked to more carefully, and re-reading John's comment, I see that we need to consider the readout times for the two types of devices in focalplanelayout.txt. It looks like we can't pick a SIM_VISTIME that is shorter than the readout time for the CCD devices (as opposed to CMOS), which is 3.0 sec in data/lsst/focalplanelayout.txt, otherwise exptime would become negative, and a negative value would crash the number of photon Poisson draw for each source.

— Reply to this email directly or view it on GitHubhttps://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/144#issuecomment-190072056.

[jchiang87]

We're using v3.4.2, which has

exptime=(vistime-(nsnap-1)*devvalue)/nsnap;

so we can only pick a small, non-negative exposure time if vistime > devvalue for nsnap=2.

[johnrpeterson]

yes, if you want to do 2 snaps, you are right. so if you want to do 0.1 second exposures, pick the visit time of 3.2 seconds.

john

On Feb 29, 2016, at 10:53 AM, James Chiang notifications@github.com<mailto:notifications@github.com> wrote:

We're using v3.4.2, which has

exptime=(vistime-(nsnap-1)*devvalue)/nsnap;

so we can only pick a small, non-negative exposure time if vistime > devvalue for nsnap=2.

— Reply to this email directly or view it on GitHubhttps://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/144#issuecomment-190266353.

[drphilmarshall]

Excellent, thanks John. Is there a downside that you can see to choosing a visit time of 3.000002 seconds for this test? Would every source in the instance catalog get asked for a photon in two such comically short exposures?

On Mon, Feb 29, 2016 at 9:42 AM, johnrpeterson notifications@github.com wrote:

yes, if you want to do 2 snaps, you are right. so if you want to do 0.1 second exposures, pick the visit time of 3.2 seconds.

john

On Feb 29, 2016, at 10:53 AM, James Chiang <notifications@github.com mailto:notifications@github.com> wrote:

We're using v3.4.2, which has

exptime=(vistime-(nsnap-1)*devvalue)/nsnap;

so we can only pick a small, non-negative exposure time if vistime > devvalue for nsnap=2.

— Reply to this email directly or view it on GitHub< https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/144#issuecomment-190266353>.

— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/144#issuecomment-190305499 .

[SimonKrughoff]

Is there even any reason to do two snaps? I find it easier to wrangle data if I keep it set to 1 snap. We also don't have anything more than a naive snap combination step.

[johnrpeterson]

yeah, you could certainly do that, or just change the two snaps to one as simon said.

by the way, the reason this is so complicated is to correctly simulate guiders which don’t have a readout time and take many exposures during the same time.

john

On Feb 29, 2016, at 12:54 PM, Phil Marshall notifications@github.com<mailto:notifications@github.com> wrote:

Excellent, thanks John. Is there a downside that you can see to choosing a visit time of 3.000002 seconds for this test? Would every source in the instance catalog get asked for a photon in two such comically short exposures?

On Mon, Feb 29, 2016 at 9:42 AM, johnrpeterson notifications@github.com<mailto:notifications@github.com> wrote:

yes, if you want to do 2 snaps, you are right. so if you want to do 0.1 second exposures, pick the visit time of 3.2 seconds.

john

On Feb 29, 2016, at 10:53 AM, James Chiang notifications@github.com<mailto:notifications@github.com mailto:notifications@github.com> wrote:

We're using v3.4.2, which has

exptime=(vistime-(nsnap-1)*devvalue)/nsnap;

so we can only pick a small, non-negative exposure time if vistime > devvalue for nsnap=2.

— Reply to this email directly or view it on GitHub< https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/144#issuecomment-190266353>.

— Reply to this email directly or view it on GitHub https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/144#issuecomment-190305499 .

— Reply to this email directly or view it on GitHubhttps://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/144#issuecomment-190309416.

[TomGlanzman]

A slot in the Twinkles-phoSimII pipeline has been created but at the moment does nothing. Given the reliability of phoSim (no crashes in Run 1!), this is not at the top of my priority list. But I think it is worth keeping the issue alive.

[drphilmarshall]

Hey @TomGlanzman Does our PhoSim smoketest now do something useful? Do you plan to alter it before Run 3, or can we close this out? (We close issues so we don't have to look at them, the closed issues are still easily found by searching or browsing.)

[TomGlanzman]

There continues to be discussion on this topic. Let's leave the issue open for a while longer.