Chandrima-04 commented 1 year ago

Hi,

I have installed metapi using micromamba. Now am trying to run the code, but getting this error:

`'/pbtech_mounts/homes051/chb4004/micromamba/envs/metapi/lib/python3.11/site-packages/metapi' wrong header: Index(['short_forward_reads', 'short_reverse_reads'], dtype='object') Running metapi mag_wf: snakemake --snakefile /pbtech_mounts/homes051/chb4004/micromamba/envs/metapi/lib/python3.11/site-packages/metapi/snakefiles/mag_wf.smk --configfile ./config.yaml --cores 240 --until all --rerun-incomplete --keep-going --printshellcmds --reason --use-conda --conda-prefix ~/.conda/envs --profile ./profiles/slurm --local-cores 8 --jobs 30

Real running cmd: snakemake --snakefile /pbtech_mounts/homes051/chb4004/micromamba/envs/metapi/lib/python3.11/site-packages/metapi/snakefiles/mag_wf.smk --configfile ./config.yaml --cores 240 --until all --rerun-incomplete --keep-going --printshellcmds --reason --use-conda --conda-prefix ~/.conda/envs --profile ./profiles/slurm --local-cores 8 --jobs 30 `

I had initialized my folder using metapi init -d . -s metapi.tsv

This is my slurm script: `#!/bin/sh

SBATCH --nodes=1

SBATCH --ntasks-per-node=1

SBATCH --cpus-per-task=4

SBATCH --time=168:00:00

SBATCH --mem=25GB

SBATCH --job-name=metapi

SBATCH --error metapi--%j.err

micromamba shell hook -s bash eval "$(~/bin/micromamba shell hook -s bash)" micromamba activate metapi

metapi mag_wf all --use-conda --run-remote`

This is my sample file. The header seems to be correct based on documentation. Can you let me know what is going wrong? Is there an example code/more documentation available?

metapi - Sheet1.csv

alienzj commented 1 year ago

Hi, @Chandrima-04,

I found that the latest documentation doesn't align with metapi v2.5.0. Now I released metapi v3.0.0, you can install latest metapi by runing: mamba install -c bioconda -c conda-forge metapi=3.0.0. Then the error about headers will dispear.

By the way, if you want to run metapi on remote server, there is no need to add metapi mag_wf all --use-conda --run-remote to SLURM script. You just run metapi mag_wf all --use-conda --run-remote in login node, then snakemake will help you to submit the jobs.

If you want to adjust any information about SLURM job, such as partition, qos, take a look at <working_folder>/profiles/slurm/cluster.yaml.

Please let me know if you have any questions.

Chandrima-04 commented 11 months ago

do you have the zenodo file with all the databases and scripts you used (the ones that will be input to config.yaml)? I have been running in login node but this is the error

Building DAG of jobs... MissingInputException in rule rmhost_bowtie2_index in file /pbtech_mounts/homes051/chb4004/micromamba/envs/metapi/lib/python3.11/site-packages/metapi/snakefiles/../rules/rmhost.smk, line 205: Missing input files for rule rmhost_bowtie2_index: output: /home/jiezhu/databases/genomics/human/CHM13/chm13v2.0_plusY/chm13v2.0.fa.1.bt2, /home/jiezhu/databases/genomics/human/CHM13/chm13v2.0_plusY/chm13v2.0.fa.2.bt2, /home/jiezhu/databases/genomics/human/CHM13/chm13v2.0_plusY/chm13v2.0.fa.3.bt2, /home/jiezhu/databases/genomics/human/CHM13/chm13v2.0_plusY/chm13v2.0.fa.4.bt2, /home/jiezhu/databases/genomics/human/CHM13/chm13v2.0_plusY/chm13v2.0.fa.rev.1.bt2, /home/jiezhu/databases/genomics/human/CHM13/chm13v2.0_plusY/chm13v2.0.fa.rev.2.bt2 affected files: /home/jiezhu/databases/genomics/human/CHM13/chm13v2.0_plusY/chm13v2.0.fa

alienzj commented 11 months ago

Hi, @Chandrima-04 Not yet, I have no plan to redistribute these databases.

To improve reproducibility, I will update code to let metapi download and build database automicatlly from the original source.

For hot fix, may I know can you download the database by yourselft? CHM13 can be downloaded from here: https://genome-idx.s3.amazonaws.com/bt/chm13.draft_v1.0_plusY.zip

Chandrima-04 commented 11 months ago

Thanks, can you also share the link of the other dbs? I saw in config.yaml, there are multiple files needed.