search

njcuk9999 / apero-drs

A PipelinE to Reduce Observations - The DRS for SPIRou (CFHT)
MIT License
12 stars 0 forks source link

0.7.289-stable-test error: "name already used as a name or title" #757

Closed larnoldgithub closed 4 months ago

larnoldgithub commented 4 months ago

Neil this is an error I got running the new 0.7.289 in the container. Note that I didn't get this error last week when I did a first successfull run. I wanted to redo a test, same, from scratch.

I rebuilt the container, meaning I got the code from github etc. I did a full reset to clean up everything on disk but the raw directory with the nights.

Every is fine with the PP, I start to have errors at the EXTTELL and EXTOBJ steps: This is an example of the errors I get:

09:21:30.819-  |PROC| ***************************************************************************
09:21:30.827-@!|PROC| W[40-503-00019]: Error found for ID='1356'
09:21:30.834-@!|PROC|   apero_extract_spirou.py 2020-06-30 2497599o_pp.fits --crunfile=mini_run2.ini --program=EXTOBJ[01356] --recipe_kind=extract-science --shortname=EXTOBJ --parallel=True
09:21:30.842-  |PROC| ***************************************************************************
09:21:30.850-  |PROC| 
 E[01-002-00012]: Error <class 'ValueError'>: name already used as a name or
        title
         file = '/apero-data/test_helu/asset/telluric/tapas_all_sp.fits.gz' 
         function = drs_table.py.read_table()
09:21:30.859-  |PROC| 
09:21:30.866-  |PROC| 
09:21:30.874-  |PROC| ***************************************************************************
njcuk9999 commented 4 months ago

I did "fix" the column names (as you mentioned they were incorrect) maybe I broke something (I didn't run a mini data run as I though the change didn't require it) I will look into this.

njcuk9999 commented 4 months ago

If you were really motivated you could copy the tapas_all_sp.fits.gz from version 0.7.288 over that one and see if it works - that would speed up my job when I come to this problem.

njcuk9999 commented 4 months ago

Okay it seems to be that I indeed screwed up naming the columns and now have two columns called "trans_ch4" I'll figure out which is which and correct the v0.7.289 branch

njcuk9999 commented 4 months ago

okay I've updated v0.7.289-live and v0.7.289-stable-test so a simple git pull should fix this problem (hopefully I renamed the correct trans_ch4 column)

larnoldgithub commented 4 months ago

I'm running a new test. rebuilt everything from scratch. Container version.

The aim of this test is to check (that the bug is gone) that I do get the same results that last week.

I'm planning a second test after that: I'mm process the mini run in 2 steps: REF and CALIB first, then clean the red/ and the continue with the EXTTEL and EXTOBJ etc. just to test the ability of Apero for work with only the calib files from the calib/

One question still: when Apero builds the products, does it pick up the blaze, wave etc from calib/ or from red/ ?

larnoldgithub commented 4 months ago

the mini_run2.ini ended with no error. So the tapas name bug is fixed.

njcuk9999 commented 4 months ago

perfect thanks for verifying! I've been too busy to run a mini test so its very helpful!

njcuk9999 commented 4 months ago

One question still: when Apero builds the products, does it pick up the blaze, wave etc from calib/ or from red/ ?

In theory everything including the postprocessing should only ever use files from the calib and telluric directory... I'm not sure about the lbl codes - I can't remember if I added a way to "copy" the science/calibration data to lbl or whether it is always just symlinked - if it is always symlinked then apero-lbl stuff may break (or run on a smaller data set) -- I'm really not sure at this point