FTFIT2 errors during the second tellu correction (0.7.288)

[larnoldgithub]

This is not a mySQL timeout error. I got a few like this one: Apparently a math issue not well handled.

17:29:54.776- |FTELLU| 17:29:54.783- |FTELLU| 17:29:54.783- |FTELLU| SPIROU @PID-00016991189937151648-VIX3 (V0.7.288) 17:29:54.783- |FTELLU| py3.9.16 git:v0.7.288-stable-test 17:29:54.783- |FTELLU| 17:29:54.789- |FTELLU| 17:29:54.795- |FTELLU| █████╗ ██████╗ ███████╗██████╗ ██████╗
17:29:54.800- |FTELLU| ██╔══██╗██╔══██╗██╔════╝██╔══██╗██╔═══██╗ 17:29:54.806- |FTELLU| ███████║██████╔╝█████╗ ██████╔╝██║ ██║ 17:29:54.811- |FTELLU| ██╔══██║██╔═══╝ ██╔══╝ ██╔══██╗██║ ██║ 17:29:54.817- |FTELLU| ██║ ██║██║ ███████╗██║ ██║╚██████╔╝ 17:29:54.823- |FTELLU| ╚═╝ ╚═╝╚═╝ ╚══════╝╚═╝ ╚═╝ ╚═════╝
17:29:54.828- |FTELLU| * 17:29:55.848-|FTFIT2[55507]| DRS Setup: 17:29:55.854-|FTFIT2[55507]| 17:29:55.855-|FTFIT2[55507]| DRS_ROOT: /apero/apero-drs/apero 17:29:55.855-|FTFIT2[55507]| DRS_DATA_RAW: /data/spirou4/arlinks 17:29:55.855-|FTFIT2[55507]| DRS_DATA_REDUC: /data/spirou4/apero-data/offline/red 17:29:55.855-|FTFIT2[55507]| DRS_DATA_WORKING: /data/spirou4/apero-data/offline/tmp 17:29:55.856-|FTFIT2[55507]| DRS_CALIB_DB: /data/spirou4/apero-data/offline/calib 17:29:55.856-|FTFIT2[55507]| DRS_TELLU_DB: /data/spirou4/apero-data/offline/tellu 17:29:55.856-|FTFIT2[55507]| DRS_DATA_ASSETS: /data/spirou4/apero-data/offline/asset 17:29:55.856-|FTFIT2[55507]| DRS_DATA_MSG: /data/spirou4/apero-data/offline/msg 17:29:55.856-|FTFIT2[55507]| DRS_DATA_RUN: /data/spirou4/apero-data/offline/run 17:29:55.857-|FTFIT2[55507]| DRS_DATA_PLOT: /data/spirou4/apero-data/offline/plot 17:29:55.857-|FTFIT2[55507]| DRS_CONFIG: /apero/config/offline/user_config.ini 17:29:55.857-|FTFIT2[55507]| DRS_CONFIG: /apero/config/offline/user_constants.ini 17:29:55.857-|FTFIT2[55507]| DRS_CONFIG: apero.constants.default.default_config 17:29:55.857-|FTFIT2[55507]| DRS_CONFIG: apero.constants.default.default_keywords 17:29:55.858-|FTFIT2[55507]| DRS_CONFIG: config.instruments.spirou.default_config.py 17:29:55.858-|FTFIT2[55507]| DRS_CONFIG: config.instruments.spirou.default_keywords.py 17:29:55.858-|FTFIT2[55507]| DRS_CONFIG: core.instruments.default.default_constants.py 17:29:55.858-|FTFIT2[55507]| DRS_CONFIG: core.instruments.spirou.default_constants.py 17:29:55.858-|FTFIT2[55507]| DATABASE: MYSQL 17:29:55.858-|FTFIT2[55507]| DATABASE-calib: aperooffline@localhost:calib_offline_db 17:29:55.859-|FTFIT2[55507]| DATABASE-tellu: aperooffline@localhost:tellu_offline_db 17:29:55.859-|FTFIT2[55507]| DATABASE-findex: aperooffline@localhost:findex_offline_db 17:29:55.859-|FTFIT2[55507]| DATABASE-log: aperooffline@localhost:log_offline_db 17:29:55.859-|FTFIT2[55507]| DATABASE-astrom: aperooffline@localhost:astrom_offline_db 17:29:55.859-|FTFIT2[55507]| DATABASE-lang: aperooffline@localhost:lang_offline_db 17:29:55.859-|FTFIT2[55507]| DATABASE-reject: aperooffline@localhost:reject_offline_db 17:29:55.860-|FTFIT2[55507]| PRINT_LEVEL: all 17:29:55.860-|FTFIT2[55507]| LOG_LEVEL: all 17:29:55.860-*|FTFIT2[55507]| DRS_PLOT: 0 17:29:55.866- |FTFIT2[55507]| 17:29:55.872-|FTFIT2[55507]| Arguments used: 17:29:55.878-|FTFIT2[55507]| 17:29:55.878-|FTFIT2[55507]| --OBS_DIR: /data/spirou4/apero-data/offline/red/19BQ05-Sep25 17:29:55.878-|FTFIT2[55507]| --FILES[0]: [EXT_E2DS_FF] 2438568o_pp_e2dsff_AB.fits 17:29:55.879-|FTFIT2[55507]| --PROGRAM: FTFIT2[55507] 17:29:55.879-|FTFIT2[55507]| --RECIPE_KIND: tellu-science 17:29:55.879-|FTFIT2[55507]| --PARALLEL: True 17:29:55.879-|FTFIT2[55507]| --SHORTNAME: FTFIT2 17:29:55.879-|FTFIT2[55507]| --CRUNFILE: run_FTFIT2_only.ini 17:29:55.885- |FTFIT2[55507]| ** 17:29:57.631- |FTFIT2[55507]| Checking input qc for: files 17:29:57.647- |FTFIT2[55507]| All input files files passed QC 17:29:58.183- |FTFIT2[55507]| 17:29:58.197- |FTFIT2[55507]| Processing file 1 of 1 17:29:58.213- |FTFIT2[55507]| ***** 17:29:58.805-|FTFIT2[55507]| File validated: OBJECT=DHTAUB DPRTYPE=OBJ_DARK 17:29:58.972- |FTFIT2[55507]| Wavelength solution read from reference. 17:29:58.973- |FTFIT2[55507]| Wave file = 17:29:58.973- |FTFIT2[55507]| /data/spirou4/apero-data/offline/calib/3444961B5D_pp_e2dsff_AB_wavesol_ref_AB.fits 17:29:59.026- |FTFIT2[55507]| Wavelength solution read from infile. 17:29:59.026- |FTFIT2[55507]| Wave file = E902CCB288_pp_e2dsff_AB_wave_night_AB.fits 17:29:59.073- |FTFIT2[55507]| Loading TELLU_TEMP files 17:29:59.186- |FTFIT2[55507]| No template found in telluric database 17:29:59.219- |FTFIT2[55507]| Loading TELLU_MODEL files 17:29:59.366-|FTFIT2[55507]| Calculating Barycentric Corrections 17:29:59.389- |FTFIT2[55507]| Identified object as DHTAUB with BERV = 27.2525 km/s 17:29:59.673- |FTFIT2[55507]| Cleaning OH lines 17:30:08.088- |FTFIT2[55507]| OH Cleaning: Num of masked lines = 290 17:30:08.502- |FTFIT2[55507]| Template used for CCF computation: 17:30:08.502- |FTFIT2[55507]| /data/spirou4/apero-data/offline/asset/telluric/trans_others_abso_ccf.mas 17:30:08.581- |FTFIT2[55507]| Template used for CCF computation: 17:30:08.581- |FTFIT2[55507]| /data/spirou4/apero-data/offline/asset/telluric/trans_h2o_abso_ccf.mas 17:30:08.601- |FTFIT2[55507]| Pre-cleaning data 17:30:09.156- |FTFIT2[55507]| iteration=0 dexpo=inf dv_abso=0.000 [m/s] 17:30:09.156- |FTFIT2[55507]| expo_water=0.000 expo_others=0.000 /apero/apero-drs/apero/core/math/gen_math.py:586: RuntimeWarning: invalid value encountered in divide num = np.exp(-0.5 * (res / sig) 2) * (1 - odd_cut) /apero/apero-drs/apero/science/telluric/gen_tellu.py:924: RuntimeWarning: invalid value encountered in divide ccf_water = (ccf_water / np.polyval(water_coeffs, drange)) - 1 /apero/apero-drs/apero/science/telluric/gen_tellu.py:928: RuntimeWarning: invalid value encountered in divide ccf_others = (ccf_others / np.polyval(others_coeffs, drange)) - 1 17:30:09.223- |FTFIT2[55507]| amp_water=nan amp_others=nan 17:30:09.235-!!|FTFIT2[55507]| E[01-010-00001]: Unhandled error has occurred: Error <class 'IndexError'> 17:30:09.235-!!|FTFIT2[55507]| 17:30:09.236-!!|FTFIT2[55507]| Traceback (most recent call last): 17:30:09.236-!!|FTFIT2[55507]| File "/apero/apero-drs/apero/core/utils/drs_startup.py", line 433, in run 17:30:09.236-!!|FTFIT2[55507]| llmain = func(recipe, params) 17:30:09.236-!!|FTFIT2[55507]| File "/apero/apero-drs/bin/apero_fit_tellu_spirou.py", line 240, in main 17:30:09.236-!!|FTFIT2[55507]| tpreprops = telluric.tellu_preclean(params, recipe, infile, wprops, 17:30:09.237-!!|FTFIT2[55507]| File "/apero/apero-drs/apero/science/telluric/gen_tellu.py", line 1112, in tellu_preclean 17:30:09.237-!!|FTFIT2[55507]| snr_others = -amp_others_arr[0] / mp.estimate_sigma(ccf_others) 17:30:09.237-!!|FTFIT2[55507]| IndexError: index 0 is out of bounds for axis 0 with size 0 17:30:09.237-!!|FTFIT2[55507]| 17:30:09.544-|FTFIT2[55507]| 17:30:09.567-@!|FTFIT2[55507]| W[40-003-00005]: Recipe apero_fit_tellu_spirou has NOT been successfully 17:30:09.567-@!|FTFIT2[55507]| completed 17:30:09.583-|FTFIT2[55507]|

[larnoldgithub]

I was also surprised to see errors with targets FAKETARGET GY91M378 and NOWHERE while these targets already are in the reject_list of the apero_objdb @CharlesCadieux ?

[njcuk9999]

I'll take a look at this issue with Etienne, you are correct it shouldn't be crashing like this.

The reject list does not stop someone trying to process an object name - it just doesn't take astrometric parameters (like RA, Dec etc) from the astrometric database (apero_objdb).

[njcuk9999]

The main issue here is a duplicate of #719 - definitely needs a better error message but there is an underlying problem here.