search

njcuk9999 / apero-drs

A PipelinE to Reduce Observations - The DRS for SPIRou (CFHT)
MIT License
12 stars 0 forks source link

Errors after FTFIT1 (0.7.286) #719

Open larnoldgithub opened 1 year ago

larnoldgithub commented 1 year ago

I have 19 similar errors like this one. These observations seem to be all at low SNR, pre-cleaning fails. Looks like a bug with an index value in a vector ? Maybe other obs of those stars are tellu corrected so a correct template can be built and the odometer below will get tellu corrected then ?

03:45:21.693- |FTFIT1[20117]| * 03:45:21.698-|FTFIT1[20117]| Arguments used: 03:45:21.704-|FTFIT1[20117]| 03:45:21.704-|FTFIT1[20117]| --OBS_DIR: /data/spirou4/apero-data/offline/reduced/18BQ08-Sep24 03:45:21.705-|FTFIT1[20117]| --FILES[0]: [EXT_E2DS_FF] 2305744o_pp_e2dsff_AB.fits 03:45:21.705-|FTFIT1[20117]| --PROGRAM: FTFIT1[20117] 03:45:21.705-|FTFIT1[20117]| --RECIPE_KIND: tellu-science 03:45:21.705-|FTFIT1[20117]| --PARALLEL: True 03:45:21.705-|FTFIT1[20117]| --SHORTNAME: FTFIT1 03:45:21.706-*|FTFIT1[20117]| --CRUNFILE: run_MK_to_FTFIT1_only.ini 03:45:21.711- |FTFIT1[20117]| 03:45:24.085- |FTFIT1[20117]| Checking input qc for: files 03:45:24.135- |FTFIT1[20117]| All input files files passed QC 03:45:24.411- |FTFIT1[20117]| 03:45:24.457- |FTFIT1[20117]| Processing file 1 of 1 03:45:24.468- |FTFIT1[20117]| 03:45:24.801-|FTFIT1[20117]| File validated: OBJECT=HST10 DPRTYPE=OBJ_DARK 03:45:24.916- |FTFIT1[20117]| Wavelength solution read from reference. 03:45:24.916- |FTFIT1[20117]| Wave file = 03:45:24.916- |FTFIT1[20117]| /data/spirou4/apero-data/offline/calibDB/3444961B5D_pp_e2dsff_AB_wavesol_ref_AB.fits 03:45:24.974- |FTFIT1[20117]| Wavelength solution read from infile. 03:45:24.975- |FTFIT1[20117]| Wave file = E1A15FE630_pp_e2dsff_AB_wave_night_AB.fits 03:45:25.037- |FTFIT1[20117]| Loading TELLU_TEMP files 03:45:25.093- |FTFIT1[20117]| No template found in telluric database 03:45:25.118- |FTFIT1[20117]| Loading TELLU_MODEL files 03:45:25.203-|FTFIT1[20117]| Calculating Barycentric Corrections 03:45:25.219- |FTFIT1[20117]| Identified object as HST10 with BERV = 25.7507 km/s 03:45:25.397- |FTFIT1[20117]| Cleaning OH lines 03:45:33.799- |FTFIT1[20117]| OH Cleaning: Num of masked lines = 290 03:45:34.086- |FTFIT1[20117]| Template used for CCF computation: 03:45:34.086- |FTFIT1[20117]| /data/spirou4/apero-data/offline/assets/telluric/trans_others_abso_ccf.mas 03:45:34.171- |FTFIT1[20117]| Template used for CCF computation: 03:45:34.171- |FTFIT1[20117]| /data/spirou4/apero-data/offline/assets/telluric/trans_h2o_abso_ccf.mas 03:45:34.191- |FTFIT1[20117]| Pre-cleaning data 03:45:34.775- |FTFIT1[20117]| iteration=0 dexpo=inf dv_abso=0.000 [m/s] 03:45:34.776- |FTFIT1[20117]| expo_water=0.000 expo_others=0.000 /apero/apero-drs/apero/core/math/gen_math.py:586: RuntimeWarning: invalid value encountered in divide num = np.exp(-0.5 * (res / sig) 2) * (1 - odd_cut) /apero/apero-drs/apero/science/telluric/gen_tellu.py:918: RuntimeWarning: invalid value encountered in divide ccf_water = (ccf_water / np.polyval(water_coeffs, drange)) - 1 /apero/apero-drs/apero/science/telluric/gen_tellu.py:922: RuntimeWarning: invalid value encountered in divide ccf_others = (ccf_others / np.polyval(others_coeffs, drange)) - 1 03:45:34.845- |FTFIT1[20117]| amp_water=nan amp_others=nan 03:45:34.857-!!|FTFIT1[20117]| E[01-010-00001]: Unhandled error has occurred: Error <class 'IndexError'> 03:45:34.858-!!|FTFIT1[20117]| 03:45:34.858-!!|FTFIT1[20117]| Traceback (most recent call last): 03:45:34.858-!!|FTFIT1[20117]| File "/apero/apero-drs/apero/core/utils/drs_startup.py", line 432, in run 03:45:34.858-!!|FTFIT1[20117]| llmain = func(recipe, params) 03:45:34.858-!!|FTFIT1[20117]| File "/apero/apero-drs/bin/apero_fit_tellu_spirou.py", line 240, in main 03:45:34.859-!!|FTFIT1[20117]| tpreprops = telluric.tellu_preclean(params, recipe, infile, wprops, 03:45:34.859-!!|FTFIT1[20117]| File "/apero/apero-drs/apero/science/telluric/gen_tellu.py", line 1106, in tellu_preclean 03:45:34.859-!!|FTFIT1[20117]| snr_others = -amp_others_arr[0] / mp.estimate_sigma(ccf_others) 03:45:34.859-!!|FTFIT1[20117]| IndexError: index 0 is out of bounds for axis 0 with size 0 03:45:34.859-!!|FTFIT1[20117]| 03:45:35.043-**|FTFIT1[20117]| ***** 03:45:35.055-@!|FTFIT1[20117]| W[40-003-00005]: Recipe apero_fit_tellu_spirou has NOT been successfully 03:45:35.055-@!|FTFIT1[20117]| completed 03:45:35.067-**|FTFIT1[20117]| ***

njcuk9999 commented 1 year ago

Yer so looks like it expects amp_others_arr to be a list with at least one entry... this error would suggest there are no entries in amp_others_arr.

Seems like the only way this can happen is if the ccf contains a NaN (there is a break that stops the iterative process) I'm not sure why this is but it is probably worth plotting the input file (wavelength vs flux)

Its probably also related to the warning you get before this point:

/apero/apero-drs/apero/core/math/gen_math.py:586: RuntimeWarning: invalid value encountered in divide
num = np.exp(-0.5 * (res / sig) ** 2) * (1 - odd_cut)
/apero/apero-drs/apero/science/telluric/gen_tellu.py:918: RuntimeWarning: invalid value encountered in divide
ccf_water = (ccf_water / np.polyval(water_coeffs, drange)) - 1
/apero/apero-drs/apero/science/telluric/gen_tellu.py:922: RuntimeWarning: invalid value encountered in divide
ccf_others = (ccf_others / np.polyval(others_coeffs, drange)) - 1

Which probably means there is a large section of NaNs I'm not sure why though

larnoldgithub commented 1 year ago

After FTFIT2 I got the same 19 files failing as in FTFIT1 - expected results. There are some NaN sections indeed. Faint object/low SNR obs are impacted. Let me know if you want a e2dsff file to explore, I can easily make one available for you. Here are some screenshots, if it helps. 19AQ02-Feb16-2375373o_pp_e2dsff_AB fits Screenshot 2023-09-11 at 17 38 39 Screenshot 2023-09-11 at 17 45 24

njcuk9999 commented 1 year ago

Yep I think we would need the e2dsff file going in to this. Maybe we have the file so we just need the odometer code?

larnoldgithub commented 1 year ago

Here are the 19 files that failed: 02:36:58.392-@!|PROC| apero_fit_tellu_spirou.py 23AQ11-May02 2871701o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[10071] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:36:58.667-@!|PROC| apero_fit_tellu_spirou.py 23AQ01-Feb04 2846291o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[11047] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:36:58.926-@!|PROC| apero_fit_tellu_spirou.py 22BQ11-Nov16 2812847o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[13292] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:36:59.212-@!|PROC| apero_fit_tellu_spirou.py 18BQ08-Sep24 2305744o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[20117] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:36:59.523-@!|PROC| apero_fit_tellu_spirou.py 20AQ08-May10 2487789o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[50841] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:36:59.728-@!|PROC| apero_fit_tellu_spirou.py 20AQ08-May10 2487788o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[50842] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:36:59.884-@!|PROC| apero_fit_tellu_spirou.py 20AQ02-Feb18 2470180o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[51632] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:37:00.102-@!|PROC| apero_fit_tellu_spirou.py 20AQ02-Feb18 2470179o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[51633] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:37:00.360-@!|PROC| apero_fit_tellu_spirou.py 20AQ02-Feb18 2470178o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[51634] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:37:00.619-@!|PROC| apero_fit_tellu_spirou.py 19AQ10-May13 2412953o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[57976] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:37:00.861-@!|PROC| apero_fit_tellu_spirou.py 19AQ10-May13 2412952o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[57977] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:37:01.103-@!|PROC| apero_fit_tellu_spirou.py 19AQ10-May13 2412951o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[57978] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:37:01.345-@!|PROC| apero_fit_tellu_spirou.py 19AQ10-May13 2412950o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[57979] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:37:01.603-@!|PROC| apero_fit_tellu_spirou.py 19BQ05-Sep25 2438571o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[58109] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:37:01.862-@!|PROC| apero_fit_tellu_spirou.py 19BQ05-Sep25 2438570o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[58110] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:37:02.104-@!|PROC| apero_fit_tellu_spirou.py 19BQ05-Sep25 2438569o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[58111] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:37:02.379-@!|PROC| apero_fit_tellu_spirou.py 19AQ02-Feb16 2375375o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[61125] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:37:02.638-@!|PROC| apero_fit_tellu_spirou.py 19AQ02-Feb16 2375374o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[61126] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True 02:37:02.880-@!|PROC| apero_fit_tellu_spirou.py 19AQ02-Feb16 2375373o_pp_e2dsff_AB.fits --crunfile=run_MK_to_FTFIT1_only.ini --program=FTFIT1[61127] --recipe_kind=tellu-science --shortname=FTFIT1 --parallel=True

larnoldgithub commented 11 months ago

re-running with the .288 ended with the same errors for the same files.

larnoldgithub commented 11 months ago

20AQ02-Feb18 2470180o_pp_e2dsff_AB.fits is a SLS target, so you should the data.

njcuk9999 commented 7 months ago

I'm looking at 2305744o_pp_e2dsff_AB.fits (for us in directory 2018-09-25)

Seems the spectrum looks like this:

Image

More importantly most of the object "flux" is negative:

Image

There is a line in the code (current L759 in science/telluric/gen_tellu.py) that remove flux below zero:

    # for numerical stability, remove NaNs. Setting to zero biases a bit
    # the CCF, but this should be OK after we converge
    spectrum[~np.isfinite(spectrum)] = 0.0
    spectrum[spectrum < 0.0] = 0.0

This essentially breaks the ccf (the ccf will only look at those spikes and miss most the flux). I'll talk to @eartigau about this! Maybe a background issue.

Note I looked at the file 20AQ02-Feb18 2470180o_pp_e2dsff_AB.fits (for us in directory 2020-02-09) and we don't have this problem for this file - so maybe a difference in the calibrations calculated/used?