search

njcuk9999 / apero-drs

A PipelinE to Reduce Observations - The DRS for SPIRou (CFHT)
MIT License
12 stars 0 forks source link

SKIP_FTFIT2 and SKIP_FTTEMP2 (0.7.286) #720

Open larnoldgithub opened 1 year ago

larnoldgithub commented 1 year ago

I have run FTFIT for the first time, no template (number of cores 25) After that I ran apero FTTEMP1 for to calculate the template (cores only 10) Next step is FTFIT2 (again 25cores). The reason I ran these steps separatelty is to be able to tune th number of cores.

But I ran FTFIT2, apero was skipping all commands like this one apero_fit_tellu_spirou.py 21AQ17-Jul24 2624730o_pp_e2dsff_AB.fits because the SKIP_FTFIT2 is at True by default in the complete_run.ini file. I set it to False, and will do the same for FTTEMP2 .

Maybe SKIP_FTFIT2 and SKIP_FTTEMP2 should be set to False by default in the complete_run.ini file ?

larnoldgithub commented 1 year ago

@clairem789 @andrescarmona - you may have already crossed this 'issue' - not really an issue but something to be aware of.

njcuk9999 commented 1 year ago

So the run.ini files are just examples, you do not have to use them as they are, you can create as many as you like! (But be aware that updates are not applied to custom run.ini files)

For an actual complete run (where you work out everything to do before files are created) this is not an issue, so actually we could set every recipe to SKIP_XXX = False - I'd rather keep them all True or all False though.

But you are correct that apero will see any run that uses the same recipe + has the same arguments as the same. This is hard to fix and anyone doing "custom" runs (where you break things up into bits) should probably be aware of this with all steps. I'll try to think if I could skip based on the shortname aswell (e.g. FTTEMP1 and FTTEMP2) .

larnoldgithub commented 1 year ago

Thanks Neil. I agree all SKIP* sets to false when running complete_run.ini would make it.