Asking python how many CPUs there are does not respect the amount the cluster has actually allocated to the job, causing too many processes to be spawned. Instead, we let the R script be configured on the command line:
Rscript sth_fitting.R 10
To run with 10 jobs. Also added some basic logging that shows progress every 10 jobs completed with time taken.
Asking python how many CPUs there are does not respect the amount the cluster has actually allocated to the job, causing too many processes to be spawned. Instead, we let the R script be configured on the command line:
To run with 10 jobs. Also added some basic logging that shows progress every 10 jobs completed with time taken.