shinobu384 commented 9 months ago

Dear Author I have manually installed mintie 0.4.2 and setup ref w/o error. But, mintie -w -p test_params.txt cases/.fastq.gz controls/.fastq.gz provoked error as follows. I would appreciate your help. mntieTest.3rdtime.log

╒══════════════════════════════════════════════════════════════════════════════════════════════════╕ | Starting Pipeline at 2024-02-02 21:04 | ╘══════════════════════════════════════════════════════════════════════════════════════════════════╛

================================ Stage fastq_dedupe (allvars-case) ================================= Reads before: 423500 allvars-case/temp1.fastq Reads after: 423400 allvars-case/allvars-case.1.fastq

==================================== Stage trim (allvars-case) ===================================== TrimmomaticPE: Started with arguments: -threads 2 -phred33 allvars-case/allvars-case.1.fastq.gz allvars-case/allvars-case.2.fastq.gz allvars-case/trim1.fastq /dev/null allvars-case/trim2.fastq /dev/null LEADING:20 TRAILING:20 MINLEN:80 Input Read Pairs: 105850 Both Surviving: 105850 (100.00%) Forward Only Surviving: 0 (0.00%) Reverse Only Surviving: 0 (0.00%) Dropped: 0 (0.00%) TrimmomaticPE: Completed successfully

================================== Stage assemble (allvars-case) =================================== bash: line 1: 1461 Segmentation fault /mnt/d/mintie_240129/MINTIE-0.4.2240129/MINTIE-0.4.2/tools/bin/soapdenovotrans pregraph -s config.config -o outputGraph$k -K $k -p 2 ERROR: stage assemble failed: Command in stage assemble failed with exit status = 139 :

rlens=gunzip -c allvars-case/trim1.fastq.gz allvars-case/trim2.fastq.gz | awk -v mrl=80 'BEGIN {minlen = mrl; maxlen = 0} { if (NR % 4 == 2) { rlen = length($1) ; if (rlen > maxlen) {maxlen = rlen} if (rlen < minlen) {minlen = rlen} }} END {print minlen" "maxlen}' ; min_rlen=${rlens% } ; max_rlen=${rlens# } ; if [ ! -d allvars-case/SOAPassembly ]; then mkdir allvars-case/SOAPassembly ; fi ; cd allvars-case/SOAPassembly ; echo "max_rd_len=$max_rlen" > config.config ; echo -e "[LIB]\nq1=../../allvars-case/trim1.fastq.gz\nq2=../../allvars-case/trim2.fastq.gz" >> config.config ; if [ -e SOAP.fasta ]; then rm SOAP.fasta ; fi ; for k in 79 49 ; do if [ $k -gt $min_rlen ]; then echo "WARNING: Kmer size $k exceeds minimum read length ${min_rlen}. Please double check parameters." ; else /mnt/d/mintie_240129/MINTIE-0.4.2240129/MINTIE-0.4.2/tools/bin/soapdenovotrans pregraph -s config.config -o outputGraph$k -K $k -p 2 ; /mnt/d/mintie_240129/MINTIE-0.4.2240129/MINTIE-0.4.2/tools/bin/soapdenovotrans contig -g outputGraph$k ; cat outputGraph$k.contig | sed "s/^>/>k${k}/g" >> SOAP.fasta ; fi ; done ; cd ../../ ; /mnt/d/mintie_240129/MINTIE-0.4.2_240129/MINTIE-0.4.2/tools/bin/dedupe in=allvars-case/SOAPassembly/SOAP.fasta out=stdout.fa threads=2 overwrite=true | /mnt/d/mintie_240129/MINTIE-0.4.2_240129/MINTIE-0.4.2/tools/bin/fasta_formatter | awk '!/^>/ { next } { getline seq } length(seq) > 100 { print $0 "\n" seq }' > allvars-case/allvars-case_denovo_filt.fasta ; if [ ! -s allvars-case/allvars-case_denovo_filt.fasta ] ; then rm allvars-case/allvars-case_denovo_filt.fasta ; echo "ERROR: de novo assembled contigs fasta file is empty." ; echo "Please check paths for SOAPdenovoTrans, dedupe and fasta" ; echo "formatter are correct, and their dependencies are installed." ; fi ;

========================================= Pipeline Failed ==========================================

In stage Unknown: One or more parallel stages aborted. The following messages were reported:

----------------------------------- assemble ( allvars-case ) ------------------------------------

Command in stage assemble failed with exit status = 139 :

rlens=gunzip -c allvars-case/trim1.fastq.gz allvars-case/trim2.fastq.gz | awk -v mrl=80 'BEGIN {minlen = mrl; maxlen = 0} { if (NR % 4 == 2) { rlen = length($1) ; if (rlen > maxlen) {maxlen = rlen} if (rlen < minlen) {minlen = rlen} }} END {print minlen" "maxlen}' ; min_rlen=${rlens% } ; max_rlen=${rlens# } ; if [ ! -d allvars-case/SOAPassembly ]; then mkdir allvars-case/SOAPassembly ; fi ; cd allvars-case/SOAPassembly ; echo "max_rd_len=$max_rlen" > config.config ; echo -e "[LIB]\nq1=../../allvars-case/trim1.fastq.gz\nq2=../../allvars-case/trim2.fastq.gz" >> config.config ; if [ -e SOAP.fasta ]; then rm SOAP.fasta ; fi ; for k in 79 49 ; do if [ $k -gt $min_rlen ]; then echo "WARNING: Kmer size $k exceeds minimum read length ${min_rlen}. Please double check parameters." ; else /mnt/d/mintie_240129/MINTIE-0.4.2240129/MINTIE-0.4.2/tools/bin/soapdenovotrans pregraph -s config.config -o outputGraph$k -K $k -p 2 ; /mnt/d/mintie_240129/MINTIE-0.4.2240129/MINTIE-0.4.2/tools/bin/soapdenovotrans contig -g outputGraph$k ; cat outputGraph$k.contig | sed "s/^>/>k${k}/g" >> SOAP.fasta ; fi ; done ; cd ../../ ; /mnt/d/mintie_240129/MINTIE-0.4.2_240129/MINTIE-0.4.2/tools/bin/dedupe in=allvars-case/SOAPassembly/SOAP.fasta out=stdout.fa threads=2 overwrite=true | /mnt/d/mintie_240129/MINTIE-0.4.2_240129/MINTIE-0.4.2/tools/bin/fasta_formatter | awk '!/^>/ { next } { getline seq } length(seq) > 100 { print $0 "\n" seq }' > allvars-case/allvars-case_denovo_filt.fasta ; if [ ! -s allvars-case/allvars-case_denovo_filt.fasta ] ; then rm allvars-case/allvars-case_denovo_filt.fasta ; echo "ERROR: de novo assembled contigs fasta file is empty." ; echo "Please check paths for SOAPdenovoTrans, dedupe and fasta" ; echo "formatter are correct, and their dependencies are installed." ; fi ;

Use 'bpipe errors' to see output from failed commands.

/mnt/d/mintie_240129/MINTIE-0.4.2_240129/MINTIE-0.4.2# bpipe errors

============================== Found 1 failed commands from run 1318 ===============================

====================================== Command assemble (26) =======================================

Command : rlens=`gunzip -c allvars-case/trim1.fastq.gz allvars-case/trim2.fastq.gz | awk -v mrl=80 'BEGIN {minlen = mrl; maxlen = 0} { if (NR % 4 == 2) { rlen = length($1) ; if (rlen > maxlen) {maxlen = rlen} if (rlen < minlen) {minlen = rlen} }} END {print minlen" "maxlen}'` ; min_rlen=${rlens% } ; max_rlen=${rlens# } ; if [ ! -d allvars-case/SOAPassembly ]; then mkdir allvars-case/SOAPassembly ; fi ; cd allvars-case/SOAPassembly ; echo "max_rd_len=$max_rlen" > config.config ; echo -e "[LIB]\nq1=../../allvars-case/trim1.fastq.gz\nq2=../../allvars-case/trim2.fastq.gz" >> config.config ; if [ -e SOAP.fasta ]; then rm SOAP.fasta ; fi ; for k in 79 49 ; do if [ $k -gt $min_rlen ]; then echo "WARNING: Kmer size $k exceeds minimum read length ${min_rlen}. Please double check parameters." ; else /mnt/d/mintie_240129/MINTIE-0.4.2240129/MINTIE-0.4.2/tools/bin/soapdenovotrans pregraph -s config.config -o outputGraph$k -K $k -p 2 ; /mnt/d/mintie_240129/MINTIE-0.4.2240129/MINTIE-0.4.2/tools/bin/soapdenovotrans contig -g outputGraph$k ; cat outputGraph$k.contig | sed "s/^>/>k${k}/g" >> SOAP.fasta ; fi ; done ; cd ../../ ; /mnt/d/mintie_240129/MINTIE-0.4.2_240129/MINTIE-0.4.2/tools/bin/dedupe in=allvars-case/SOAPassembly/SOAP.fasta out=stdout.fa threads=2 overwrite=true | /mnt/d/mintie_240129/MINTIE-0.4.2_240129/MINTIE-0.4.2/tools/bin/fasta_formatter | awk '!/^>/ { next } { getline seq } length(seq) > 100 { print $0 "\n" seq }' > allvars-case/allvars-case_denovo_filt.fasta ; if [ ! -s allvars-case/allvars-case_denovo_filt.fasta ] ; then rm allvars-case/allvars-case_denovo_filt.fasta ; echo "ERROR: de novo assembled contigs fasta file is empty." ; echo "Please check paths for SOAPdenovoTrans, dedupe and fasta" ; echo "formatter are correct, and their dependencies are installed." ; fi ; Started : Fri Feb 02 21:05:10 JST 2024 Stopped : Fri Feb 02 21:05:10 JST 2024 Exit Code : 139 Config: Name | Value

      procs                   | 8
      memory                  | 180
      max_per_command_threads | 16
      executor                | local
      mem_param               | mem
      name                    | assemble
      proc_mode               | 1
      usePollerFileWatcher    | true
      walltime                | 20:00:00
      queue                   | batch
      concurrency             | 16

Output :

    bash: line 1:  1461 Segmentation fault      /mnt/d/mintie_240129/MINTIE-0.4.2_240129/MINTIE-0.4.2/tools/bin/soapdenovotrans pregraph -s config.config -o outputGraph_$k -K $k -p 2

mcmero commented 9 months ago

Are you running on Mac OS? I seem to remember this might be an issue with the install script for SOAPdenovoTrans. I would check if your soapdenovotrans installation runs independently of MINTIE.

I'd also strongly recommend installing via bioconda instead, which addresses these kinds of installation issues.

shinobu384 commented 9 months ago

thank you mcmero I try to run mintie on windows 11 WSL2. As per your recommendation, I installed the mintie via conda and set up ref, ending up as follows. How can this be fixed? many thanks.

install

cd /mnt/c/Users/PC_User/Desktop/mintieBymamba_240205 mamba create -c conda-forge -c bioconda -n mintieBymamba_240205 mintie 2>&1 | tee install.log # /root/miniconda3/envs/mintieBymamba_240130 conda activate mintieBymamba_240205

setup ref

mintie -r 2>&1 | tee setupRef.log gzip: chess3.0.gtf already exists; do you wish to overwrite (y or n)? # y

Checking that all required references were setup: genome_fasta looks like it has been setup tx_annotation looks like it has been setup trans_fasta looks like it has been setup WARNING: ann_info could not be found!!!! You will need to setup ann_info manually, then add its path to references.groovy tx2gene looks like it has been setup gmap_refdir looks like it has been setup gmap_genome looks like it has been setup

WARNING: One or more command did not install successfully. See warning messages above. You will need to correct this before running MINTIE.

cat ${mintie_dir}/share/mintie-0.4.2-0/tools.groovy // Path to tools used by the MINTIE pipeline bpipe="bpipe" fastuniq="fastuniq" dedupe="dedupe.sh" trimmomatic="trimmomatic" fasta_formatter="fasta_formatter" samtools="samtools" bedtools="bedtools" soapdenovotrans="SOAPdenovo-Trans-127mer" salmon="salmon" gmap="gmap" R="R" python="python"

mcmero commented 9 months ago

It looks like your exon reference generation failed for some reason. Find your ${mintie_dir}/share/mintie-0.4.2-0/ directory, cd into it, load your mintie conda env and try running:

python util/make_exon_reference.py ref/chess3.0.gtf

shinobu384 commented 9 months ago

Thanks mcmero But another error came up. Sorry to ask you again, but let me know how to fix this.

cd ${mintie_dir}/share/mintie-0.4.2-0/ conda activate mintieBymamba_240205 python util/make_exon_reference.py ref/chess3.0.gtf

Traceback (most recent call last): File "util/make_exon_reference.py", line 14, in from pybedtools import BedTool File "/root/miniconda3/envs/mintieBymamba_240205/lib/python2.7/site-packages/pybedtools/init.py", line 18, in from . import contrib File "/root/miniconda3/envs/mintieBymamba_240205/lib/python2.7/site-packages/pybedtools/contrib/init.py", line 3, in from . import venn_maker File "/root/miniconda3/envs/mintieBymamba_240205/lib/python2.7/site-packages/pybedtools/contrib/venn_maker.py", line 12, in from pybedtools import helpers File "/root/miniconda3/envs/mintieBymamba_240205/lib/python2.7/site-packages/pybedtools/helpers.py", line 13, in import pysam File "/root/miniconda3/envs/mintieBymamba_240205/lib/python2.7/site-packages/pysam/init.py", line 5, in from pysam.libchtslib import * ImportError: libhts.so.2: cannot open shared object file: No such file or directory

cat ${mintie_dir}/share/mintie-0.4.2-0/references.groovy

// Path to references used by the MINTIE pipeline gmap_refdir="/root/miniconda3/envs/mintieBymamba_240205/share/mintie-0.4.2-0/ref/" genome_fasta="/root/miniconda3/envs/mintieBymamba_240205/share/mintie-0.4.2-0/ref/hg38.fa" tx_annotation="/root/miniconda3/envs/mintieBymamba_240205/share/mintie-0.4.2-0/ref/chess3.0.gtf" trans_fasta="/root/miniconda3/envs/mintieBymamba_240205/share/mintie-0.4.2-0/ref/chess3.0.fa" ann_info="" tx2gene="/root/miniconda3/envs/mintieBymamba_240205/share/mintie-0.4.2-0/ref/tx2gene.txt" gmap_refdir="/root/miniconda3/envs/mintieBymamba_240205/share/mintie-0.4.2-0/ref" gmap_genome="gmap_genome""

mcmero commented 8 months ago

In your mintie environment, can you check where your python executable is being invoked with which python, and then check the version with python --version. You should be seeing python 3.x, not 2.7.

If you're seeing the latter I would try completely removing the environment and creating a fresh install with

mamba create -n mintie2 -c conda-forge -c bioconda mintie python=3

shinobu384 commented 8 months ago

success! Thank you so much.

Oshlack / MINTIE

Error in Testing MINTIE #39

install

setup ref

cat ${mintie_dir}/share/mintie-0.4.2-0/references.groovy