search

metagenome-atlas / atlas

ATLAS - Three commands to start analyzing your metagenome data
https://metagenome-atlas.github.io/
BSD 3-Clause "New" or "Revised" License
364 stars 97 forks source link

Error in rule #686

Closed robbueck closed 1 year ago

robbueck commented 1 year ago

I'm trying to run atlas on SGE. I added the SGE options in the ~/.config/snakemake/cluster/key_mapping.yaml so they look like this:

# only parameters defined in key_mapping (see below) are passed to the command in the order specified.
system: "SGE" #check if system is defined below

SGE:
  command: 'qsub'
  key_mapping:
    name: "-N"
    threads: "-pe smp {}"
    mem_mb: "-l m_mem_free={}M"
    account: "-A {}"
    queue: "-q {}"
    time_min: "-l h_rt={}"

but I get following errors:

CLUSTER: 2023-07-26 17:47:24 Automatically choose best queue to submit
CLUSTER: 2023-07-26 17:47:25 Choose queue: longrun
CLUSTER: 2023-07-26 17:47:25 submit command: qsub -N -pe smp 4 -l h_rt=5760 -l m_mem_free=1000M -q longrun /data/study/fastq_files/.snakemake/tmp.xne35nac/snakejob.initialize_qc.202.sh
CLUSTER: 2023-07-26 17:47:25 Job can't be submitted
Unable to read script file because of error: error opening smp: No such file or directory

The submit command looks right, but I don't understand the issue with -pe smp {}, as this is the way I usually request the number of cores.

I also would like to omit the -q option, as the scheduler in the cluster does this automatically depending on the resources. How can I do this?

Not sure if this is relevant, but the directory ~/.config/snakemake/cluster that I created with 

cookiecutter --output-dir ~/.config/snakemake https://github.com/metagenome-atlas/clusterprofile.git
cluster
SGE

contains two files: pbs_status.py and slurm_status.py, but nothing called SGE_status.py, although ~/.config/snakemake/cluster/config.yaml contains the line 

cluster-status: "SGE_status.py" #
SilasK commented 1 year ago

I suggest you to look at the profile for sge. https://github.com/Snakemake-Profiles/sge/tree/master/

It has already rhe sge submit, status and cancel script.

You might need to modify it a little bit but I think it should be easier than with my profile..

robbueck commented 1 year ago

I just discovered this was already an issue #327 cookiecutter https://github.com/metagenome-atlas/clusterprofile.git --checkout sge worked fine for me after correcting a formatting error in key_mapping.yaml