Error: "./eli_download_docs_launcher.sh: line 3: sbatch: command not found"

facebookresearch / ELI5

Scripts and links to recreate the ELI5 dataset.

Other

316 stars 40 forks source link

Error: "./eli_download_docs_launcher.sh: line 3: sbatch: command not found" #17

Closed gdgovinddaga6 closed 4 years ago

gdgovinddaga6 commented 4 years ago

cd slurm_scripts ./eli_download_docs_launcher.sh

I have successfully downloaded the dataset and while running this command, I'm facing some issues. Please help me out, Thanks in advance.

yjernite commented 4 years ago

Do you have access to a SLURM managed cluster? If so the sbatch command should be available, and this seems to be an issue with your cluster. If not, what kind of cluster are you using?

gdgovinddaga6 commented 4 years ago

I don't have SLURM managed cluster, How do I install cluster in my ubuntu 18.04?

yjernite commented 4 years ago

Unfortunately the current method does not support using a single machine, as it would take several weeks to run.

gdgovinddaga6 commented 4 years ago

I also wanted to know in detail Information about the configurations for the cluster to run on the cloud. Things like,

How many Nodes?
How much memory should I allocate to the SSD or HDD and RAM?
Do I need a GPU in training and preprocessing steps? If YES, what type of GPU and the model and also the size?
How much capacity should my CPU processing capacity?
Which cloud do you suggest?

And also please specify which aren't specified above or which I might miss?

Thanks in Advance.