Closed sonyahanson closed 9 years ago
Can you paste in the command line or script you used, and what the output filenames were?
This was the runscript:
#!/bin/tcsh #PBS -l walltime=72:00:00 #PBS -j oe #PBS -k oe #PBS -q batch #PBS -l procs=64,mem=64gb,vmem=64gb,pmem=64gb #PBS -V #PBS -N ensembler #PBS -M [EMAIL ADDRESS] #PBS -m abe cd $PBS_O_WORKDIR mpirun -np 64 ensembler build_models mpirun -np 64 ensembler refine_implicit
The files for which there were 64 were the build_models-meta0.yaml
files.
It sounds like all 64 processes were launched thinking they were MPI ID 0 (the master).
As mentioned in #5, this might have been due to using incompatible MPI implementations, which are hopefully sorted out now!
Sorted.
Made lots of meta yaml files (64 to be exact). Not sure why. Worked fine using an interactive session. Note that I was using 64 cores for 36 models, which could have contributed.