Errors after the reduction of all calibrations: apero_thermal_spirou (0.7.286)

larnoldgithub commented 1 year ago

Very few errors. I got two errors with apero_thermal_spirou.py 22BQ11-Nov12 and 22BQ13-Dec01:

I just re-played the commands apero_thermal_spirou.py 22BQ11-Nov12 2812091d_pp.fits 2812092d_pp.fits 2812228d_pp.fits 2812229d_pp.fits --crunfile=run_FullCalib_only.ini --program=THERM_T[04502] --recipe_kind=calib-night-T --shortname=THERM_T --parallel=True

and

apero_thermal_spirou.py 22BQ13-Dec01 2814550d_pp.fits 2814551d_pp.fits 2814689d_pp.fits 2814690d_pp.fits --crunfile=run_FullCalib_only.ini --program=THERM_T[04512] --recipe_kind=calib-night-T --shortname=THERM_T --parallel=True

and it worked, no error, missing file isn't missing anymore.

2 questions: 1/ what happened ? RAM doesn't seem to have been an issue, always << 100Gb allocated (around 50Gb). 2/ Do I need to reprocess everything for the calibrations of those 2 nights to be sure everything is ok ?

Thanks

njcuk9999 commented 1 year ago

This one I'm not sure about. What is the DPRTYPE for 161C3D015B_pp_e2dsff_AB.fits and AC7554BC8C_pp_e2dsff_AB.fits?

Looks like the extraction code broke the first time and didn't produce the e2ds files (but then you should have an error relating to this), even weird that after running it again it worked.

larnoldgithub commented 1 year ago

Both 161C3D015B_pp_e2dsff_AB.fits and AC7554BC8C_pp_e2dsff_AB.fits are DPRTYPE DARK_DARK_TEL

larnoldgithub commented 1 year ago

full log of one on the errors:

Why does the routine build AC7554BC8C_pp.fits and then immediately try to read AC7554BC8C_pp_e2dsff_AB.fits ?

njcuk9999 commented 1 year ago

It says in the log message:

02:07:55.295- |THERM_T[04512]|A[40-016-00021]: Found and reading extracted file /data/spirou4/apero-data/offline/reduced/22BQ13-Dec01/AC7554BC8C_pp_e2dsff_AB.fits
02:07:55.358-!!|THERM_T[04512]|E[01-001-00012]: Read Error: File 'AC7554BC8C_pp_e2dsff_AB.fits' could not be found in directory '/data/spirou4/apero-data/offline/reduced/22BQ13-Dec01'.

If it finds it on disk it doesn't try extracting it again, it may mean there is an inconsistency between your database and on disk.

Does this file exist on disk? Is it in the file index database?

njcuk9999 commented 1 year ago

Actually I just looked up the code, it has nothing to do with the database.

it does os.path.exists('/data/spirou4/apero-data/offline/reduced/22BQ13-Dec01/AC7554BC8C_pp_e2dsff_AB.fits')

So there is something weird going on with that file. Can you:

check it exists on disk
try opening it with dfits
try opening it in python ( using astropy and fits.open)

Maybe the file got corrupted some how?

larnoldgithub commented 1 year ago

File exists on disk I can read it with dfits and ds9 I can open it in python and display the e2dsff Screenshot 2023-08-29 at 21 46 29

larnoldgithub commented 1 year ago

When is the file reduced/22BQ13-Dec01/AC7554BC8C_pp_e2dsff_AB.fits created during the whole calibration process ?

larnoldgithub commented 1 year ago

The processing so far: reset, precheck, preprocessing of all files ( no errors), then all calibraion processing, including REF night. I'll now go for EXTALL.

njcuk9999 commented 1 year ago

Okay weird, the e2ds file is created as part of the thermal code itself (if the file doesn't already exist on disk).

The code clearly states that it finds the e2ds on disk at an earlier stage (otherwise it would remake the data), but then the problem is these lines of code:

    if not os.path.exists(filename):
        # check directory exists
        dirname = os.path.dirname(filename)
        if not os.path.exists(dirname):
            eargs = [dirname, os.path.basename(filename), func_name]
            WLOG(params, 'error', textentry('01-001-00013', args=eargs))
        else:
            eargs = [os.path.basename(filename), dirname, func_name]
            WLOG(params, 'error', textentry('01-001-00012', args=eargs))

We see that it is going into errro 01-001-00012 which should mean that the file does not exist

i.e. the lines:

os.path.exists(filename) returns False
os.path.exists(os.path.dirname(filename)) returns True

Could you try these two lines above in python? (with filename= /data/spirou4/apero-data/offline/reduced/22BQ13-Dec01/AC7554BC8C_pp_e2dsff_AB.fits) and tell me the return (True or False) for both?

larnoldgithub commented 1 year ago

Don't forget I re-did the apero_thermal_spirou.py for these 2 nights and got no error. I don't know how this may bias the analysis we are doing now.

os.path.exists(filename) True os.path.exists(os.path.dirname(filename)) True

njcuk9999 commented 1 year ago

Oh okay, maybe the file was corrupted?

njcuk9999 / apero-drs

Errors after the reduction of all calibrations: apero_thermal_spirou (0.7.286) #712