search

LSSTScienceCollaborations / StackClub

Learning the LSST software Stack, by writing jupyter notebook tutorials.
https://stackclub.readthedocs.io/
MIT License
45 stars 17 forks source link

Update Notebooks to v20.0.0 #236

Closed kadrlica closed 4 years ago

kadrlica commented 4 years ago

Hi @drphilmarshall @djperrefort @stargaser @ih64 @brantr @jtmyles @kadrlica @jbkalmbach,

It's that time again... v20.0.0 was released recently, and one or more of your notebooks are now failing :( I've run beavis-ci and the failing notebooks can be found here: https://stackclub.readthedocs.io/en/latest/notebooks.html

I think many of these failures are related to the deprecation of the Twinkles data repo. While it is probably technically easy to switch over to another data set, many of the notebooks are hand-tuned to specific sources or regions of images, which means it will be some human work to make the switch. @SimonKrughoff has suggested that we move to the HSC re-run data set (i.e., /datasets/hsc/repo/rerun/RC/w_2020_14/DM-24359-sfm or similar). I'll be going through these notebooks during the Stack Club sessions on Friday, but I'd certainly appreciate help.

I'd like to target the PCW in early August for these updates.

SimonKrughoff commented 4 years ago

To be clear, there is a new rerun every couple of weeks, so we should also try to keep up with the newer ones. Most of the time, this should be as simple as just changing which rerun the notebook points to. There will also be a day when only gen3 reruns are available, and we'll need to keep an eye out for that too.

kadrlica commented 4 years ago

We've only committed to update at every major release. Will the rerun repos stick around on disk long enough?

SimonKrughoff commented 4 years ago

Very good question. I have asked, and the statement is that reprocessing runs will be saved for a year, which should be enough. That said, we should identify the reprocessing that went with v20 and use that when updating these notebooks.

SimonKrughoff commented 4 years ago

OK. The HSC rerun for v20 is located at /datasets/hsc/repo/rerun/RC/v20_0_0_rc1.

Edit: I'm informed that should stick around until at least June 2021.

jbkalmbach commented 4 years ago

Should we maybe just replace my Twinkles notebook with the notebook from the Stack Club lesson? I think it has everything from the original light curve notebook and more.

kadrlica commented 4 years ago

@jbkalmbach sounds like a good plan to me.

kadrlica commented 4 years ago

Simon and I talked about data sets, and we agreed that /datasets/hsc/repo/rerun/RC/v20_0_0_rc1 is the one to use. We also discussed whether it would be useful to define a variable in stackclub so that in the future we can update this path in just one place.

kadrlica commented 4 years ago

Thanks everyone! I've been able to build with beavis using v20 and everything passes! https://stackclub.readthedocs.io/en/latest/notebooks.html

I'm going to declare success and close this issue.