annelopes
commented
4 years ago Hi all,
two modifications:
-
we created a directory in /shared/projects/meetu2019/data where all data to be shared with everyone will be stored. I also put the small test db in this directory. I consequently edited the paths in the test scripts provided in /shared/projects/meetu2019/runtest/*slurm , and in the README.
-
in fact, we ask you to write your outputs in /shared/projects/meetu2019/username/ not in your home. For those who have given their emails, you should have received an email from IFB (with additional explanations) once your account is created.
Have a nice week-end,
A.
elolaine
Hello,
we prepared some scripts with a README to help you to start with the cluster.
step1:
1st warning: Keep in mind that the IFB cluster is used by many colleagues in France, so you have to pay attention to everything you do. In particular, the cluster consists of what we call a "master" and "nodes". The master interacts with the nodes.
Basically, you send your jobs from the master, and the latter sends your jobs on the available nodes. The master is shared by everyone, so again, be careful. Do not execute expensive calculations directly on the master. If you want to make some preliminary tests or some basic but relatively expensive tasks (such as copy/tar a big file...), please do it on a node via what we call an interactive job (more details at https://ifb-elixirfr.gitlab.io/cluster/doc/sinteractive/).
Then for all the other jobs, please use the SLURM system (basic explanations in the quick start guide - see above). The quick start guide should be enough for you to start (information on how to launch/cancel/follow a job...). Once you become an expert (!), you can have a look at https://ifb-elixirfr.gitlab.io/cluster/doc/slurm_user_guide/ for more details :)
Again, we insist on the fact that all your jobs must be submitted with the SLURM system with the command "sbatch" (see the quick start guide and our README).
step2:
ssh username@core.cluster.france-bioinformatique.frYou are now in your home directory that is: /shared/home/username/
You can have a look at /shared/bank where you will find many databases available (nr, uniprot, uniref...) for all users (including other students and researchers).
Then take a look at /shared/projects/meetu2019 All data related to Meet-U are available there. HOMSTRAD db, foldrec and query files.
Also we put 3 examples to run your first blast for example but then you can modify these scripts to run any type of jobs/scripts. So we provided 3 query files in fasta format and 3 SLURM scripts to train yourself + a README.
Of course, there are a lot of possibilities with the SLURM system, but we recommend that you start with these examples, then prepare your own examples and try to run your own jobs.
preparing/sending data on the cluster To copy your data on the cluster:
scp mydata username@core.cluster.france-bioinformatique.fr:Do not forget the ":" at the end! (only for the "scp" not for the "ssh"). For sending data on the cluster, opt for sending an zip archive rather than many files or a directory.preparing an archive of fasta files for example:
tar cfvz data.tgz *fastauntar and unzip the archive (tar file)tar -xzvf data.tgzDo not forget the "-" before the args to untar, but keep it to tar.If you want to go for more complex jobs, please have a look to the IFB documentation or send an issue! Also, the next session (next week) will be an opportunity to review your scripts together. So we recomend you to take advantage of this session to test your scripts with us!
2nd warning: (!) Keep in mind that the IFB cluster is used by many colleagues in France, so you have to pay attention to everything you do.
++
Anne
PS: please put your mails in your folders so that we can create your accounts