Running BGCFlow in DTU LSF HPC

[matinnuhamunada]

For DTU students/staff that wanted to run BGCFlow on the LSF HPC facility, follow these steps:

• Access the HPC by following the guide from here https://www.hpc.dtu.dk/?page_id=2501

# go to login node
ssh userid@login1.gbar.dtu.dk # access to login node

• install miniforge on the login node: https://github.com/conda-forge/miniforge#mambaforge

• Get extra space by requesting a scratch dir by emailing the HPC support at support@cc.dtu.dk, https://www.hpc.dtu.dk/?page_id=927

# once you have been assigned a scratch dir, create a symlink to your home dir
SCRATCH_DIR="/work3/<user_id>/" # change user id accordingly

# create a symlink to the scratch dir
ln -s $SCRATCH_DIR drive

• install BGCFlow on the login node: https://github.com/NBChub/bgcflow?tab=readme-ov-file#quick-start
• create a symlink to BGCFlow on the scratch dir

  cd drive/
  mkdir bgcflow
  cd bgcflow
  # we keep the config and workflow in the home directory as it is being backed up
  ln -s ~/bgcflow/workflow/ workflow
  ln -s ~/bgcflow/config/ config

  which results to something like this:

  
  gbarlogin1(matinnu) $ tree
  .
  ├── config -> /zhome/b2/0/153431/bgcflow/config/
  └── workflow -> /zhome/b2/0/153431/bgcflow/workflow/

2 directories, 0 files (base) ~/drive/bgcflow

- Install the lsf plugin

linuxsh # go to one of the worker node

conda run -n bgcflow mamba install bioconda::snakemake-executor-plugin-lsf -y

- execute the workflow

Create profile config file:
```yaml
jobs: 4
executor: lsf
default-resources:
    mem_mb: 200
#set-threads:
#    myrule: 5
set-resources:
    prokka:
        mem: 8000MB
    antismash:
        mem: 8000MB
    checkm:
        mem: 16000MB
    automlst_wrapper:
        mem: 8000MB
    bigscape:
        mem: 24000MB
    arts:
        mem: 8000MB

Then run the workflow

# IMPORTANT, run this from the worker node uslsf_project=<project name> lsf_queue=<hpc>ng linuxsh 
cd ~/drive/bgcflow/
conda run -n bgcflow snakemake --executor lsf --use-conda --profile <path to a directory containing the profile config.yaml> --default-resources  lsf_project=<project name> lsf_queue=<hpc>

to find the right queue, use: bqueues -u <user id>

• The final structure will look like this:

  
  gbarlogin1(matinnu) $ (cd ~ && tree bgcflow/ drive/ -L 2)
  bgcflow/
  ├── CITATION.cff
  ├── config
  │   ├── config.yaml
  │   ├── Lactobacillus_delbrueckii
  │   └── lanthipeptide_lactobacillus
  ├── Dockerfile
  ├── envs.yaml
  ├── LICENSE
  ├── profiles
  │   └── config.yaml
  ├── README.md
  ├── resources
  │   └── README.md
  └── workflow
  ├── Alleleome
  ├── BGC
  ├── bgcflow
  ├── Database
  ├── envs
  ├── lsabgc
  ├── Metabase
  ├── misc
  ├── notebook
  ├── ppanggolin
  ├── report
  ├── Report
  ├── rules
  ├── rules_bgc.yaml
  ├── rules_ppanggolin.yaml
  ├── rules.yaml
  ├── schemas
  ├── scripts
  └── Snakefile
  drive/
  └── bgcflow
  ├── config -> /zhome/b2/0/153431/bgcflow/config/
  └── workflow -> /zhome/b2/0/153431/bgcflow/workflow/

17 directories, 19 files (base) ~

[matinnuhamunada]

The worker node does not allow downloading files using FTP access, to fix this edit the main Snakefile so that rule ncbi_genome_download is done on the parent node:

report: "report/workflow.rst"

localrules: all, ncbi_genome_download
...

[matinnuhamunada]

Seems like some jobs fail because it exceed memory usage limit. Need to fix this in the LSF profile.