search

roryk / ipython-cluster-helper

Tool to easily start up an IPython cluster on different schedulers.
148 stars 23 forks source link

Batch and log file destination #36

Open nspies opened 8 years ago

nspies commented 8 years ago

Is there any way to direct where the batch submission and log files get placed? If I specify a large number of jobs, my working directory gets filled with a huge number of these files.

roryk commented 8 years ago

Hi @nspies,

Sorry there isn't a way to specify where the batch files end up. They should all have the same prefix though so you can easily delete them with a sge_* or something similar.

nspies commented 8 years ago

Alright, thanks. BTW, is this something that ipyparallel is controlling? I see that the batch_file_name argument gets passed on to from ipyparallel.apps import launcher but that's as far as I've gone. I can file a ticket there if it makes more sense to do that.

roryk commented 8 years ago

Hi @nspies,

We generate the batch_file_name, so could theoretically append a working directory name to it, but I'm not 100% sure how the different schedulers will resolve where the working directory is, if it is where the batch file is or where the batch file is submitted from.