search

LSSTDESC / imSim

GalSim based Rubin Observatory image simulation package
https://lsstdesc.org/imSim
BSD 3-Clause "New" or "Revised" License
36 stars 15 forks source link

add option to disable IERS downloads #427

Closed jchiang87 closed 10 months ago

cwwalter commented 11 months ago

I don't think I see the file:: URL to tell it where to get the local file in this case. Does it go somewhere else, or not checked in?

jchiang87 commented 11 months ago

Doing

iers.conf.auto_download = False
iers.conf.iers_degraded_accuracy = 'warn'

disables the download entirely, so there doesn't need to be any URLs specified and we don't need to keep local copies of the IERS corrections and leap seconds files around.

jchiang87 commented 11 months ago

Unfortunately, I just found out for the multiprocessing runs, we need to have these lines called for each process, in addition to the main process. Adding the lines to batoid_wcs.py seems to work.

cwwalter commented 11 months ago

Ah OK. For my understanding: Do having the file around do anything different at all if the calculation is for the future?

jchiang87 commented 11 months ago

Do having the file around do anything different at all if the calculation is for the future?

According to the docs, the IERS download data are predictive 1 year into the future. So if we are computing for a date less than a year into the future, the most recent file would be provide numbers within the IERS accuracy spec.

jchiang87 commented 11 months ago

@jmeyers314 I added some code to batoid_wcs.py to disable the IERS downloads and also to silence those erfa warnings. Could you have a look please? Also, I see that batoid_rubin wants to download a bend.yaml file to its data/bend folder. This is causing my tests to fail when I run using a shared stack at S3DF. Those downloads may also be a problem for running on Theta in the batch queues, where remote downloads seemingly aren't allowed, but I guess they happen only in certain circumstances?

jmeyers314 commented 11 months ago

Looks good to me.

We can manually do the batoid_rubin data download somewhere appropriate instead of letting it try to auto-download.

jchiang87 commented 10 months ago

Unless I hear otherwise, I'll merge this at 2pm PT today.

cwwalter commented 10 months ago

I can imagine we want to do something where we recreate real observations after operations start.

But, as long as the errors are much smaller than the pointing accuracy this should be find. It would probably be good to have a sense of how big an effect this was.

rmjarvis commented 10 months ago

I think there is zero chance that any use case will require the iers updates. Even trying to reproduce actual observations. We don't know the actual telescope pointing well enough for this to matter. But even if I'm wrong, it's trivial for a user to get the update if they really want it.

python -c "from astropy.time import Time; Time.now().tai"

will download the necessary data if the current data is out of date. If you want, we can provide that as a standalone script people can use to get the update if they want it.

jchiang87 commented 10 months ago

Thanks everyone for helping on this one!