update survey-simulations tutorial to work with latest desisurvey/surveysim

[sbailey]

The current survey-simulations tutorial works with the old desisurvey/surveysim in the 18.11 software release, but not with the newer faster better desisurvey/surveysim in 18.12 and later. Please update the tutorial to work with the latest release.

[sbailey]

@dkirkby could you update this surveysim tutorial prior to the collaboration meeting? We'd like to have it working by the tutorial session on Monday July 8, which leads into the survey strategy session on July 9 and the hack day on July 12.

If you can't make the updates yourself, could you recruit someone who can? Thanks.

[sbailey]

I just chatted with @schlafly who agree to take a look at this. Thanks.

[schlafly]

I just went through this, and the tutorial mostly runs through how to use the now deprecated 'progress' module. @dkirkby, how do you envision the tutorial being updated? I could imagine:

• operating directly on the surveysim output files
• extending the forecast module to accommodate the features that the progress module had
• updating the progress module to accommodate whatever changes happened. Thanks!

[dkirkby]

I will take a look at this now...

[dkirkby]

I think the most useful material for next week's tutorial session would be some examples using the 100 weather realizations in $DESI_ROOT/datachallenge/surveysim2018/weather/, both individually and as an ensemble.

Each realization contains exposures.fits and stats.fits (but unfortunately no weather.fits), so this means:

• Load a single exposures file using its wrapper class.
• Make some histograms and scatter plots of the per-exposure quantities.
• Make some all-sky plots of the per-tile quantities using plots.plot_sky_passes().
• Load a single stats file using its wrapper class.
• Demonstrate SurveyStatistics.summarize() and SurveyStatistics.plot().
• Loop over the 100 weather realizations to study variability in, e.g., number of DARK tiles completed after year 4.

Thanks for taking a look at this @schlafly!

[schlafly]

Thanks David. I took a look at this. Here's what I put together https://github.com/desihub/tutorials/blob/surveysim192/survey-simulations.ipynb

I still need to remove some redundancy between what you had already done and what I added, but at least everything works now.

Some minor comments...

• Load a single exposures file using its wrapper class https://surveysim.readthedocs.io/en/latest/api.html#surveysim.exposures.ExposureList .

I didn't use the wrapper class here; it doesn't really have any useful access methods, and later accessing the relevant fields through _exposures and _tiledata felt a bit odd?

• Loop over the 100 weather realizations to study variability in, e.g., number of DARK tiles completed after year 4.

This was slightly more involved than I wanted, given the absence of copy_range, but it worked out all right. But I just queried completed exposures falling into an MJD range out of the exposures table for each run, and that looks like it worked fine.

[dkirkby]

Looks good! Some minor suggestions below:

The outputs from a survey simulation are two FITS files, one organized by tile and the other organized by exposure.

There is now one file with separate HDUs for per-tile and per-exp info (as your code shows).

tiledat = desimodel.io.load_tiles()

I recommend using tiledat = desisurvey.tiles.get_tiles() (docs) instead since this ensures you use the same tiles files consistently (and provides some utilities like airmass). Replace tiledat['ra'] with tiledat.tileRA, etc (attrs are tileID, passnum, tileRA, tileDEC, dust_factor).

I see that you use desisurvey.tiles.Tiles() later, but desisurvey.tiles.get_tiles() has the advantage of only reading the file into memory once when called several times.

Append semicolon (or plot.show()) to plot_sky_passes() lines to avoid the distracting return values.

[schlafly]

Looks good! Some minor suggestions below:

Thanks David. These are all in. I just opened a PR to merge this, and put your name on it. Thanks!

[weaverba137]

Fixed by #28