hoelzer
commented
1 year ago Hey @bharat1912, thanks for your interest in our pipeline RIBAP!
Hm, strange. Actually, we changed the way the ILPs are generated and in the newest release of RIBAP there should be no ilp_build process anymore. What I just did:
# maybe run the following command two times, sometimes I have the feeling that the first pull does not get the most up to date changes
nextflow pull hoelzer-lab/ribap
nextflow info hoelzer-lab/ribap
# do you see version 1.0.0 in the list?
nextflow run hoelzer-lab/ribap -r 1.0.0 --fasta 'data/*.fasta' -profile local,conda
# I am using as input for testing the few FASTAs in this folder: https://github.com/hoelzer-lab/ribap/tree/main/data
# which you can get via cloning this repository:
# git clone https://github.com/hoelzer-lab/ribap.git
# The terminal output for me looks like this:
N E X T F L O W ~ version 23.04.1
Launching `https://github.com/hoelzer-lab/ribap` [friendly_hamilton] DSL2 - revision: f5118310a3 [1.0.0]
Profile: local,conda
Current User: martin
Nextflow-version: 23.04.1
Starting time: 15-04-2023 06:51 UTC
Workdir location (intermediate files):
/Users/martin/git/ribap/work
Output dir name:
results
Conda cache directory:
conda
INFORMATION: ILPs and their intermediate results are deleted to save disk space (use --keepILPs to keep them).
executor > local (5)
[21/8e8462] process > RIBAP:rename (1) [100%] 5 of 5 ✔
[- ] process > RIBAP:prokka -
[- ] process > RIBAP:strain_ids -
[- ] process > RIBAP:roary -
[- ] process > RIBAP:mmseqs2 -
[- ] process > RIBAP:mmseqs2tsv -
[- ] process > RIBAP:ilp_refinement -
[- ] process > RIBAP:combine_roary_ilp -
[- ] process > RIBAP:prepare_msa -
[- ] process > RIBAP:mafft -
[- ] process > RIBAP:fasttree -
[- ] process > RIBAP:nw_display -
[- ] process > RIBAP:generate_html -
[- ] process > RIBAP:generate_upsetr_input -
[- ] process > RIBAP:upsetr -So I think that you are running an older version of RIBAP somehow. Although defining 1.0.0 as the newest version. Maybe there was a hick-up during installation. Did you install and use RIBAP before? If nothing helps, delete the code and pull a fresh version:
rm -r /home/bharat/.nextflow/assets/hoelzer-lab/ribap
nextflow pull hoelzer-lab/ribap
...For me, your error looks like you are using an older version of RIBAP and for the ilp_build process, your local Python is used where some dependency is missing/not configured:
import networkx as nx
File "/home/bharat/.local/lib/python3.6/site-packages/networkx/init.py", line 115, in
import networkx.readwrite
File "/home/bharat/.local/lib/python3.6/site-packages/networkx/readwrite/init.py", line 15, in
from networkx.readwrite.graphml import *
File "/home/bharat/.local/lib/python3.6/site-packages/networkx/readwrite/graphml.py", line 314, in
class GraphML(object):
File "/home/bharat/.local/lib/python3.6/site-packages/networkx/readwrite/graphml.py", line 344, in GraphML
types = [(np.float64, "float"), (np.float32, "float"),
AttributeError: module 'numpy' has no attribute 'float64'This should not happen when using newer versions of RIBAP and our pre-defined conda or docker environments.
In this context and in general, we also recommend using containers (docker, singularity) instead of conda because the containers are more reliable on various system configurations. But besides, the newest version f RIBAP should also work with conda.
bharat1912
I have installed nextflow as follows: $mamba create -n nextflow nextflow $conda activate nextflow
In my dir of genome fasta files (very diverse set of 79 genomes of Family Bacillaceaea), run:
$nextflow run hoelzer-lab/ribap -r 1.0.0 --fasta "$HOME/.nextflow/assets/hoelzer-lab/ribap/data/*.fasta" -profile local,conda
The process terminate at the following point: executor > local (179) [90/243e9c] process > rename (77) [100%] 79 of 79 ✔ [ac/44ede2] process > prokka (79) [100%] 79 of 79 ✔ [af/7129e6] process > strain_ids [100%] 1 of 1 ✔ [46/095b2b] process > roary (3) [ 0%] 0 of 5 [91/5b15a1] process > mmseqs2 [100%] 1 of 1 ✔ [5b/cdcb5e] process > mmseqs2tsv [100%] 1 of 1 ✔ [5c/4efa93] process > ilp_build (775) [ 0%] 0 of 2973 [- ] process > ilp_solve - [- ] process > combine_roary_ilp - [- ] process > prepare_msa - [- ] process > mafft - [- ] process > fasttree - [- ] process > nw_display - [- ] process > combine_msa - [- ] process > generate_html - [- ] process > generate_upsetr_input - [- ] process > upsetr - [- ] process > raxml - ERROR ~ Error executing process > 'ilp_build (27)'
Caused by: Process
ilp_build (27)terminated with an error exit status (1)Command executed:
parallel -j "1" 'ILP.py --max --indel {} > '"$PWD"'/{/.}.ilp 2>/dev/null' ::: "$PWD"/*tsv 2>/dev/null
ILP.py --max --indel ABAAEGMF-vs-FGGDGHCF.tsv > $(basename ABAAEGMF-vs-FGGDGHCF.tsv .tsv).ilp
BN=$(basename ABAAEGMF-vs-FGGDGHCF.tsv .tsv)
mkdir ilp ILP.py --max --indel ABAAEGMF-vs-FGGDGHCF.tsv
TMP=$(basename $PWD) mv ilp ilp_"${TMP}"
echo $BN
Command exit status: 1
Command output: (empty) executor > local (179) [90/243e9c] process > rename (77) [100%] 79 of 79 ✔ [ac/44ede2] process > prokka (79) [100%] 79 of 79 ✔ [af/7129e6] process > strain_ids [100%] 1 of 1 ✔ [- ] process > roary (3) - [91/5b15a1] process > mmseqs2 [100%] 1 of 1 ✔ [5b/cdcb5e] process > mmseqs2tsv [100%] 1 of 1 ✔ [5c/4efa93] process > ilp_build (775) [ 0%] 1 of 2993, failed: 1 [- ] process > ilp_solve - [- ] process > combine_roary_ilp - [- ] process > prepare_msa - [- ] process > mafft - [- ] process > fasttree - [- ] process > nw_display - [- ] process > combine_msa - [- ] process > generate_html - [- ] process > generate_upsetr_input - [- ] process > upsetr - [- ] process > raxml - ERROR ~ Error executing process > 'ilp_build (27)'
Caused by: Process
ilp_build (27)terminated with an error exit status (1)Command executed:
parallel -j "1" 'ILP.py --max --indel {} > '"$PWD"'/{/.}.ilp 2>/dev/null' ::: "$PWD"/*tsv 2>/dev/null
ILP.py --max --indel ABAAEGMF-vs-FGGDGHCF.tsv > $(basename ABAAEGMF-vs-FGGDGHCF.tsv .tsv).ilp
BN=$(basename ABAAEGMF-vs-FGGDGHCF.tsv .tsv)
mkdir ilp ILP.py --max --indel ABAAEGMF-vs-FGGDGHCF.tsv
TMP=$(basename $PWD) mv ilp ilp_"${TMP}"
echo $BN
Command exit status: 1
Command output: (empty)
Command error: Traceback (most recent call last): File "/home/bharat/.nextflow/assets/hoelzer-lab/ribap/bin/ILP.py", line 28, in
import networkx as nx
File "/home/bharat/.local/lib/python3.6/site-packages/networkx/init.py", line 115, in
import networkx.readwrite
File "/home/bharat/.local/lib/python3.6/site-packages/networkx/readwrite/init.py", line 15, in
from networkx.readwrite.graphml import *
File "/home/bharat/.local/lib/python3.6/site-packages/networkx/readwrite/graphml.py", line 314, in
class GraphML(object):
File "/home/bharat/.local/lib/python3.6/site-packages/networkx/readwrite/graphml.py", line 344, in GraphML
types = [(np.float64, "float"), (np.float32, "float"),
AttributeError: module 'numpy' has no attribute 'float64'
Work dir: /media/bharat/volume1/Bacillaceaea_ref_July_2023/renamed_files/ribap-work/3f/126d5a5bc308fabd2dd6b13c90b95c
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named
.command.sh-- Check '.nextflow.log' file for details
.command.sh" which is as follows:$ more .command.sh
!/bin/bash -ue
parallel -j "1" 'ILP.py --max --indel {} > '"$PWD"'/{/.}.ilp 2>/dev/null' ::: "
$PWD"/*tsv 2>/dev/null
ILP.py --max --indel ABAAEGMF-vs-FGGDGHCF.tsv > $(basename ABAAEGMF-vs-FGGDGHCF
.tsv .tsv).ilp
BN=$(basename ABAAEGMF-vs-FGGDGHCF.tsv .tsv)
mkdir ilp ILP.py --max --indel ABAAEGMF-vs-FGGDGHCF.tsv
TMP=$(basename $PWD) mv ilp ilp_"${TMP}"
echo $BN
capture process environment
set +u echo BN=${BN[@]} > .command.env
I could not find any errors related to this in the Issues (open / closed) other than disk space But I have enough disk space. Any assistance will be appreciated.
Thanks