facebookresearch / ELI5

Scripts and links to recreate the ELI5 dataset.
Other
316 stars 40 forks source link

sbatch: error: Batch job submission failed: More processors requested than permitted #18

Open ssanbu08 opened 4 years ago

ssanbu08 commented 4 years ago

Bug description During the dataset creation process, while executing ./eli_download_docs_launcher.sh , I got the following error

sbatch: error: Batch job submission failed: More processors requested than permitted

I thought this is due to less number of worker nodes I have, so I changed the first line of the script from for i in {0..99}; do sbatch --export=SCNUM=$i eli_download_docs.sbatch; done to for i in {0..1}; do sbatch --export=SCNUM=$i eli_download_docs.sbatch; done but still the error persists.

Reproduction steps I've created a SLURM cluster on Azure platform with 1 master node and 2 worker nodes

Expected behavior Any advice on setting up the cluster would be really helpful.

Logs sbatch: error: Batch job submission failed: More processors requested than permitted

Output goes here

Additional context Add any other context about the problem here. (like proxy settings, network setup, overall goals, etc.)