search

tikk3r / flocs

Containers recipes for software stacks used in LOFAR data reduction.
https://tikk3r.github.io/flocs/
GNU General Public License v3.0
6 stars 6 forks source link

final products are not saved in the generated result folder #80

Open ggurkan opened 4 months ago

ggurkan commented 4 months ago

Describe the bug The runner for LINC_calibrator produces a results_LINC_calibrator folder to save the final outcome by moving certain files from tmp.* folder to results_LINC_calibrator. However, the final products are still in one of those tmp. folders.

tikk3r commented 4 months ago

I cannot reproduce this issue myself. One of my LINC calibrator runs using a fresh copy of the runners today just finished and it copied them correctly:

lofarvwf-fsweijen@ui-01:/project/lofarvwf/Data/CasA> ls L2040573_LINC_calibrator/
flocs  job_output_linc-calibrator.txt  jobrunner.sh  LINC  logs_LINC_calibrator  mslist_LINC_calibrator.json  results_LINC_calibrator

lofarvwf-fsweijen@ui-01:/project/lofarvwf/Data/CasA> ls L2040573_LINC_calibrator/results_LINC_calibrator/
3c295_LINC_calibrator_summary.json  cal_solutions.h5  inspection  logs

I would suggest comparing the runner you used to the current 5.1.0 release and see if there are any differences. Otherwise providing the full command of how you run it might give some insight.

tikk3r commented 4 months ago

I just recalled the where this does occur: if L2040573_LINC_calibrator (in your case) already exists than indeed the tmp folder gets moved as a side effect. If you delete L2040573_LINC_calibrator before a new run it should end up as intended.

I am not sure about the best fix at the moment, as I'm not sure if deleting / overwriting that folder if it exists is always desirable.