njcuk9999 / lbl

Line by line code for radial velocity
MIT License
15 stars 3 forks source link

Timeout when running NIRPS test example #41

Open jlillo opened 7 months ago

jlillo commented 7 months ago

I get a timeout error when running the NIRPS_ESO_demo at the point when it downloads the "model wave file" from the Goettinggen ftp:

2023-11-27 11:26:41.370|G| Downloading model wave file. ftp://phoenix.astro.physik.uni-goettingen.de/HiResFITS/WAVE_PHOENIX-ACES-AGSS-COND-2011.fits

I checked the ftp address and it seems to be missing a "/data/" before the "/HiResFITS", so the complete URL should be: http://phoenix.astro.physik.uni-goettingen.de/data/HiResFITS/WAVE_PHOENIX-ACES-AGSS-COND-2011.fits

Could you please check and update the code accordingly? Thanks a lot!

njcuk9999 commented 7 months ago

I've logged into the ftp and cannot see the data directory you are talking about (indeed your link is a http not a ftp) and the following works for me without fail

wget ftp://phoenix.astro.physik.uni-goettingen.de/HiResFITS/WAVE_PHOENIX-ACES-AGSS-COND-2011.fits

Weird that it didn't work for you? Maybe they changed the directory structure and now have changed back? This definitely worked before and as far as I can tell the ftp url still works!

So if you are using the wrap file you have complete control over every constant (for reasons like this).

Until this is confirmed and fixed you can add the following to rparams and see if your new link works (@CharlesCadieux could you confirm this and fix if required?).

rparams['STELLAR_WAVE_URL'] = "ftp://phoenix.astro.physik.uni-goettingen.de/data/HiResFITS/WAVE_PHOENIX-ACES-AGSS-COND-2011.fits"

I haven't tested it but that should override the default value and allow you to continue until this has been confirmed/fixed.


For @CharlesCadieux note that this is defined per instrument so if this is confirmed you need to change each of the instruments in the lbl/instruments directory.

Note that it looks like "STELLAR_MODEL_URL" will also be affected by this.


# the wave url for the stellar models
## Default = None   SPIROU_VALUE = ftp://phoenix.astro.physik.uni-goettingen.de/HiResFITS/
STELLAR_WAVE_URL: 'ftp://phoenix.astro.physik.uni-goettingen.de/HiResFITS/'

# the wave file for the stellar models (using wget)
## Default = None   SPIROU_VALUE = WAVE_PHOENIX-ACES-AGSS-COND-2011.fits
STELLAR_WAVE_FILE: 'WAVE_PHOENIX-ACES-AGSS-COND-2011.fits'

# the stellar model url
## Default = None   SPIROU_VALUE = ftp://phoenix.astro.physik.uni-goettingen.de/HiResFITS/PHOENIX-ACES-AGSS-COND-2011/{ZSTR}{ASTR}/
STELLAR_MODEL_URL: 'ftp://phoenix.astro.physik.uni-goettingen.de/HiResFITS/PHOENIX-ACES-AGSS-COND-2011/{ZSTR}{ASTR}/'
CharlesCadieux commented 7 months ago

Hello, yes I think the URL was recently changed. Someone else noticed this bug and we simply manually copied the files to their models/ directory. I thought the server was down that day (timeout error) but as found by @jlillo, it was the URL that changed. Thanks for spotting this!

I confirm the URL has changed and we should update the branches with the missing "/data/" before the "/HiResFITS/". @njcuk9999 I will update the lbl/instruments config files, but you will need to accept these changes.

njcuk9999 commented 7 months ago

Sorry I edited my post since your message, the ftp works for me. So maybe their institution is blocking the ftp.

Please make sure you test the ftp urls with wget in Montreal before changing to a http (although I think we are using wget in the code so either should work)

CharlesCadieux commented 7 months ago

With @eartigau, we decided that it was probably better to host locally these models .fits files so that in the future, we don't depend on URL changes, ftp connection block, etc.

@njcuk9999 do you prefer to host ourselves the PHOENIX models and change the URL to an astro.umontreal.ca account?

njcuk9999 commented 7 months ago

It would be safer (in terms of controlling what people download and having a link we know is valid) but you may have to get permission from the phoenix authors.

CharlesCadieux commented 7 months ago

Ok, so as you say @njcuk9999, the following wget commands works:

wget ftp://phoenix.astro.physik.uni-goettingen.de/HiResFITS/WAVE_PHOENIX-ACES-AGSS-COND-2011.fits wget http://phoenix.astro.physik.uni-goettingen.de/data/HiResFITS/WAVE_PHOENIX-ACES-AGSS-COND-2011.fits

lbl by default uses the ftp URL but this is the second person having a Timeout error when launching lbl for the first time and downloading the PHOENIX models, so it may be working in Montreal, but not at other institutions.

jlillo commented 7 months ago

Thanks a lot. The problem is solved by simply adding the following to the wrapper to use the http instead of the ftp URL: rparams['STELLAR_WAVE_URL'] = 'http://phoenix.astro.physik.uni-goettingen.de/data/HiResFITS/' rparams['STELLAR_MODEL_URL'] = 'http://phoenix.astro.physik.uni-goettingen.de/data/HiResFITS/PHOENIX-ACES-AGSS-COND-2011/Z-0.0/' Like this the code runs smoothly. Thanks a lot for your help.