dingo-gw / dingo

Dingo: Deep inference for gravitational-wave observations
MIT License
60 stars 20 forks source link

Condor output files being overwritten #77

Open stephengreen opened 2 years ago

stephengreen commented 2 years ago

When generating a waveform dataset with condor (see dingo_generate_dataset_dag), stdout is redirected to files in condor/output/. For each job, a separate output file is created. However, each job may contain several processes (e.g., when splitting waveform generation across nodes). The output of these separate processes is all directed to the same file, which causes output from earlier processes to be overwritten by later ones.

To fix this, we should specify the output file names to include the process numbers. I am not sure how to do this, however, since the pycondor Job constructor only takes the output directory name, not the file name.