weedcentipede commented 5 years ago

Hi, I'm implementing Trans-ABySS in an SLURM HPC system, when I call the CMD in the master node the script calls abyss-pe, but for some reason when I call the CMD in some other node is calling ABYSS-P instead of abyss-pe, I tried suppresing the MPI processes in the CMD with mpi 0 but it's not working (as you can see the following out)

Trans-ABySS 2.0.1
CMD: /home/larteag7/apps/transabyss-2.0.1/transabyss --se sym-single.fq sym-interleaved.fq --pe sym-interleaved.fq --outdir /home/larteag7/my-thesis/frl-2015/transabyss-default --name transabyss-dft --threads 16 --mpi 0
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Found Trans-ABySS directory at: /home/larteag7/apps/transabyss-2.0.1
Found Trans-ABySS `bin` directory at: /home/larteag7/apps/transabyss-2.0.1/bin
Found script at: /home/larteag7/apps/transabyss-2.0.1/bin/skip_psl_self.awk
Found script at: /home/larteag7/apps/transabyss-2.0.1/bin/skip_psl_self_ss.awk
Found `abyss-pe' at /home/larteag7/.conda/envs/abyss/bin/abyss-pe
Found `MergeContigs' at /home/larteag7/.conda/envs/abyss/bin/MergeContigs
Found `abyss-filtergraph' at /home/larteag7/.conda/envs/abyss/bin/abyss-filtergraph
Found `abyss-junction' at /home/larteag7/.conda/envs/abyss/bin/abyss-junction
Found `blat' at /home/larteag7/.conda/envs/abyss/bin/blat
Found `abyss-map' at /home/larteag7/.conda/envs/abyss/bin/abyss-map
# CPU(s) available:     16
# thread(s) requested:  16
# thread(s) to use:     16
CMD: bash -euo pipefail -c 'abyss-pe graph=adj --directory=/home/larteag7/my-thesis/frl-2015/transabyss-default k=32 name=transabyss-dft E=0 e=2 c=2 j=16 transabyss-dft-1.fa transabyss-dft-1.adj q=3 se="/home/larteag7/my-thesis/frl-2015/sym-interleaved.fq /home/larteag7/my-thesis/frl-2015/sym-single.fq /home/larteag7/my-thesis/frl-2015/sym-interleaved.fq"'
make: Entering directory `/home/larteag7/my-thesis/frl-2015/transabyss-default'
mpirun -np 16 ABYSS-P -k32 -q3 -e2 -E0 -c2    --coverage-hist=coverage.hist -s transabyss-dft-bubbles.fa  -o transabyss-dft-1.fa  /home/larteag7/my-thesis/frl-2015/sym-interleaved.fq /home/larteag7/my-thesis/frl-2015/sym-single.fq /home/larteag7/my-thesis/frl-2015/sym-interleaved.fq
make: Leaving directory `/home/larteag7/my-thesis/frl-2015/transabyss-default'
ERROR: CMD ended with status code 2

Is something wrong with the CMD or the installation, Thanks in advance, Luis Alfonso.

kmnip commented 5 years ago

Hi Luis,

It looks like this is related to https://github.com/bcgsc/transabyss/issues/18.

To suppress MPI, you need to unset the environment variable, SLURM_NTASKS, before running Trans-ABySS, ie.

threads=$SLURM_NTASKS
unset $SLURM_NTASKS
transabyss --threads $threads --pe $reads1 $reads2

Don't run Trans-ABySS on the master node. You can run it within batch script for SLURM.

Hope that helps! Ka Ming

weedcentipede commented 5 years ago

Hi Ka Ming, It worked this way

threads=$SLURM_NTASKS
unset SLURM_NTASKS

Thank you!, Cheers, Luis Alfonso.

webbchen commented 4 years ago

Good afternoon

I'm trying to run transabyss with one set of paired-end read files and it seems to always identify them as two files of single end reads. The read names are suffixed with /1 and /2 resp. I tried unset $SLURM_NTASKS but apart from producing a non-fatal error it doesn't do anything.

The script:

`

!/bin/bash

SBATCH -c 1

SBATCH --mem 4G

SBATCH --nodelist=foo

SBATCH --partition=bar

SBATCH -D /tmp/annew

R1=$1 R2=$2 assembly=$3 mer=$4 output_dir=$5

srun transabyss --name ${assembly}_${mer} \ --pe $R1 $R2 \ --kmer $mer \ --outdir $output_dir \ --threads 1 \ --mpi 0 `

The command:

n=test sbatch --job-name=$n --error=$n.err --output=$n.out ~/transabysstest.sh /tmp/annew/R1.fq /tmp/annew/R2.fq assemblyname 32 /tmp/annew/transabyss_stuff_32test

Then I'm getting: Trans-ABySS 2.0.1 CMD: /projects/software/transabyss-2.0.1/transabyss-2.0.1//transabyss --name assemblyname_32 --pe /tmp/annew/R1.fq /tmp/annew/R2.fq --kmer 32 --outdir /tmp/annew/stuff_32test --threads 1 --mpi 0 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Found Trans-ABySS directory at: /projects/software/transabyss-2.0.1/transabyss-2.0.1 Found Trans-ABySS `bin` directory at: /projects/software/transabyss-2.0.1/transabyss-2.0.1/bin Found script at: /projects/software/transabyss-2.0.1/transabyss-2.0.1/bin/skip_psl_self.awk Found script at: /projects/software/transabyss-2.0.1/transabyss-2.0.1/bin/skip_psl_self_ss.awk Found `abyss-pe' at /usr/bin/abyss-pe Found `MergeContigs' at /scratch/software/abyss_2.1.5/MergePaths/MergeContigs Found `abyss-filtergraph' at /scratch/software/abyss_2.1.5/FilterGraph/abyss-filtergraph Found `abyss-junction' at /scratch/software/abyss_2.1.5/Scaffold/abyss-junction Found `blat' at /projects/software/transabyss-2.0.1/bin/blat Found `abyss-map' at /scratch/software/abyss_2.1.5/Map/abyss-map # CPU(s) available: 144 # thread(s) requested: 1 # thread(s) to use: 1 Creating output directory: /tmp/annew/stuff_32test CMD: bash -euo pipefail -c 'abyss-pe graph=adj --directory=/tmp/annew/stuff_32test k=32 name=assemblyname_32 E=0 e=2 c=2 j=1 assemblyname_32-1.fa assemblyname_32-1.adj q=3 se="/tmp/annew/R1.fq /tmp/annew/R2.fq"' make: Entering directory '/tmp/annew/stuff_32test' /usr/bin/mpirun -np 1 ABYSS-P -k32 -q3 -e2 -E0 -c2 --coverage-hist=coverage.hist -s assemblyname_32-bubbles.fa -o assemblyname_32-1.fa /tmp/annew/R1.fq /tmp/annew/R2.fq ABySS 2.0.2 ABYSS-P -k32 -q3 -e2 -E0 -c2 --coverage-hist=coverage.hist -s assemblyname_32-bubbles.fa -o assemblyname_32-1.fa /tmp/annew/R1.fq /tmp/annew/R2.fq Running on 1 processors 0: Running on host foo 0: Reading `/tmp/annew/R1.fq'...

Which suggests to me that it's not recognising the reads as paired. I'd be delighted to be wrong, though!

kmnip commented 4 years ago

HI @webbchen ,

This is the intended behavior for the unitig stage of ABySS. The paired-end reads will be aligned to the assembled unitigs afterwards.

Ka Ming

webbchen commented 4 years ago

Hi Ka Ming

Many thanks for clarifying this so quickly!

Anne

bcgsc / transabyss