search

chrisquince / STRONG

Strain Resolution ON Graphs
MIT License
44 stars 9 forks source link

Problem with mamba install modules #133

Closed jyee123 closed 1 year ago

jyee123 commented 1 year ago

The install_STRONG.sh scripts makes a mamba call: mamba env create -f $STRONG_dir/conda_env.yaml &>>$LOG

When the script gets to this step, here's the error message: warning libmamba Cache file "/mnt/strong/tools/apps/miniconda3.10/pkgs/cache/ffeee55f.json" was modified by another program bioconda/linux-64 (check zst) Checked 0.3s warning libmamba Cache file "/mnt/strong/tools/apps/miniconda3.10/pkgs/cache/2a957770.json" was modified by another program bioconda/noarch (check zst) Checked 0.1s warning libmamba Cache file "/mnt/strong/tools/apps/miniconda3.10/pkgs/cache/c484148c.json" was modified by another program warning libmamba Cache file "/mnt/strong/tools/apps/miniconda3.10/pkgs/cache/0c60671d.json" was modified by another program warning libmamba Cache file "/mnt/strong/tools/apps/miniconda3.10/pkgs/cache/ebe486dd.json" was modified by another program warning libmamba Cache file "/mnt/strong/tools/apps/miniconda3.10/pkgs/cache/bcb51836.json" was modified by another program warning libmamba Cache file "/mnt/strong/tools/apps/miniconda3.10/pkgs/cache/497deca9.json" was modified by another program warning libmamba Cache file "/mnt/strong/tools/apps/miniconda3.10/pkgs/cache/09cdf8bf.json" was modified by another program warning libmamba Cache file "/mnt/strong/tools/apps/miniconda3.10/pkgs/cache/47929eba.json" was modified by another program pkgs/main/linux-64 (check zst) Checked 0.2s warning libmamba Cache file "/mnt/strong/tools/apps/miniconda3.10/pkgs/cache/3e39a7aa.json" was modified by another program pkgs/main/noarch (check zst) Checked 0.0s warning libmamba Cache file "/mnt/strong/tools/apps/miniconda3.10/pkgs/cache/2ce54b42.json" was modified by another program pkgs/r/linux-64 (check zst) Checked 0.0s warning libmamba Cache file "/mnt/strong/tools/apps/miniconda3.10/pkgs/cache/4ea078d6.json" was modified by another program pkgs/r/noarch (check zst) Checked 0.1s [+] 0.2s anaconda/linux-64 ━━━━━━━━━━━━━━━╸━━━━━━━━━━ 0.0 B / ??.?MB @ ??.?MB/s 0.2s

>>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

Traceback (most recent call last):
  File "/mnt/strong/tools/apps/miniconda3.10/lib/python3.10/site-packages/conda/exception_handler.py", line 16, in __call__
    return func(*args, **kwargs)
  File "/mnt/strong/tools/apps/miniconda3.10/lib/python3.10/site-packages/conda_env/cli/main.py", line 49, in do_call
    exit_code = getattr(module, func_name)(args, parser)
  File "/mnt/strong/tools/apps/miniconda3.10/lib/python3.10/site-packages/conda/notices/core.py", line 123, in wrapper
    return func(*args, **kwargs)
  File "/mnt/strong/tools/apps/miniconda3.10/lib/python3.10/site-packages/conda_env/cli/main_create.py", line 163, in execute
    result[installer_type] = installer.install(
  File "/mnt/strong/tools/apps/miniconda3.10/lib/python3.10/site-packages/mamba/mamba_env.py", line 55, in mamba_install
    index = load_channels(pool, channel_urls, repos, prepend=False)
  File "/mnt/strong/tools/apps/miniconda3.10/lib/python3.10/site-packages/mamba/utils.py", line 142, in load_channels
    index = get_index(
  File "/mnt/strong/tools/apps/miniconda3.10/lib/python3.10/site-packages/mamba/utils.py", line 123, in get_index
    is_downloaded = dlist.download(api.MAMBA_DOWNLOAD_FAILFAST)
RuntimeError: Download error (23) Failed writing received data to disk/application [https://conda.anaconda.org/anaconda/noarch/repodata.json.zst]
Failure writing output to destination

$ /mnt/strong/tools/apps/miniconda3.10/bin/mamba create -f /mnt/strong/tools/apps/STRONG/conda_env.yaml

environment variables: CIO_TEST= CONDA_AUTO_UPDATE_CONDA=false CONDA_ROOT=/mnt/strong/tools/apps/miniconda3.10 CURL_CA_BUNDLE= LD_PRELOAD= MANPATH=: MODULEPATH=/etc/scl/modulefiles:/etc/scl/modulefiles:/usr/share/Modules/modulefil es:/etc/modulefiles:/usr/share/modulefiles PATH=/mnt/strong/tools/apps/miniconda3.10/bin:/home/yeej/.local/bin:/home/y eej/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin :/usr/sbin REQUESTS_CA_BUNDLE= SSL_CERT_FILE=

 active environment : None
   user config file : /home/yeej/.condarc

populated config files : /home/yeej/.condarc conda version : 23.5.0 conda-build version : not installed python version : 3.10.12.final.0 virtual packages : __archspec=1=x86_64 glibc=2.28=0 linux=4.18.0=0 __unix=0=0 base environment : /mnt/strong/tools/apps/miniconda3.10 (writable) conda av data dir : /mnt/strong/tools/apps/miniconda3.10/etc/conda conda av metadata url : None channel URLs : https://conda.anaconda.org/bioconda/linux-64 https://conda.anaconda.org/bioconda/noarch https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch https://conda.anaconda.org/anaconda/linux-64 https://conda.anaconda.org/anaconda/noarch https://conda.anaconda.org/r/linux-64 https://conda.anaconda.org/r/noarch https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /mnt/strong/tools/apps/miniconda3.10/pkgs /home/yeej/.conda/pkgs envs directories : /mnt/strong/tools/apps/miniconda3.10/envs /home/yeej/.conda/envs platform : linux-64 user-agent : conda/23.5.0 requests/2.28.1 CPython/3.10.12 Linux/4.18.0-425.3.1.el8.x86_64 rocky/8.7 glibc/2.28 UID:GID : 41154:59900 netrc file : None offline mode : False An unexpected error has occurred. Conda has prepared the above report.

Also tried miniconda3.8 with same error.

I'm using Rocky 8.7.

Any ideas how to resolve?

Thank you

Sebastien-Raguideau commented 1 year ago

Hello, Can you try again? I think I saw that before: Mamba downloads packages but flag dl as fails. Once you restart it works. Otherwise, from error message it could be issue with permission to write inside the folder?

jyee123 commented 1 year ago

Hello, If "try again" you mean run install_STRONG.sh again, I just tried with the same error message. I even tried 5 consecutive times. For me, looks like when mamba is called it fails almost immediately, meaning no packages are installed.

$ conda list

packages in environment at /mnt/strong/tools/apps/miniconda3.10:

#

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge biopython 1.81 pypi_0 pypi boltons 23.0.0 py310h06a4308_0 brotlipy 0.7.0 py310h7f8727e_1002 bzip2 1.0.8 h7b6447c_0 c-ares 1.19.1 hd590300_0 conda-forge ca-certificates 2023.5.7 hbcca054_0 conda-forge certifi 2023.5.7 pyhd8ed1ab_0 conda-forge cffi 1.15.1 py310h5eee18b_3 charset-normalizer 2.0.4 pyhd3eb1b0_0 conda 23.5.0 py310hff52083_1 conda-forge conda-content-trust 0.1.3 py310h06a4308_0 conda-package-handling 2.0.2 py310h06a4308_0 conda-package-streaming 0.7.0 py310h06a4308_0 contourpy 1.1.0 pypi_0 pypi cryptography 39.0.2 py310h34c0648_0 conda-forge cycler 0.11.0 pypi_0 pypi dill 0.3.6 pypi_0 pypi fmt 9.1.0 h924138e_0 conda-forge fonttools 4.41.0 pypi_0 pypi gfapy 1.2.3 pypi_0 pypi icu 72.1 hcb278e6_0 conda-forge idna 3.4 py310h06a4308_0 jsonpatch 1.32 pyhd3eb1b0_0 jsonpointer 2.1 pyhd3eb1b0_0 keyutils 1.6.1 h166bdaf_0 conda-forge kiwisolver 1.4.4 pypi_0 pypi krb5 1.20.1 h81ceb04_0 conda-forge ld_impl_linux-64 2.38 h1181459_1 libarchive 3.6.2 h039dbb9_1 conda-forge libcurl 8.1.2 h409715c_0 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libffi 3.4.2 h6a678d5_6 libgcc-ng 13.1.0 he5830b7_0 conda-forge libgomp 13.1.0 he5830b7_0 conda-forge libiconv 1.17 h166bdaf_0 conda-forge libmamba 1.4.7 h658169a_0 conda-forge libmambapy 1.4.7 py310h8aae740_0 conda-forge libnghttp2 1.52.0 h61bc06f_0 conda-forge libnsl 2.0.0 h7f98852_0 conda-forge libsolv 0.7.24 hfc55251_1 conda-forge libsqlite 3.42.0 h2797004_0 conda-forge libssh2 1.11.0 h0841786_0 conda-forge libstdcxx-ng 13.1.0 hfd8a6a1_0 conda-forge libuuid 2.38.1 h0b41bf4_0 conda-forge libxml2 2.11.4 h0d562d8_0 conda-forge libzlib 1.2.13 hd590300_5 conda-forge lz4-c 1.9.4 hcb278e6_0 conda-forge lzo 2.10 h516909a_1000 conda-forge mamba 1.4.7 py310h51d5547_0 conda-forge matplotlib 3.7.2 pypi_0 pypi multiprocess 0.70.14 pypi_0 pypi ncurses 6.4 h6a678d5_0 networkx 3.1 pypi_0 pypi numpy 1.25.1 pypi_0 pypi openssl 3.1.1 hd590300_1 conda-forge packaging 23.0 py310h06a4308_0 pandas 2.0.3 pypi_0 pypi pathos 0.3.0 pypi_0 pypi patsy 0.5.3 pypi_0 pypi pillow 10.0.0 pypi_0 pypi pip 23.0.1 py310h06a4308_0 pluggy 1.0.0 py310h06a4308_1 pox 0.3.2 pypi_0 pypi ppft 1.7.6.6 pypi_0 pypi progressbar2 4.3b0 pypi_0 pypi pybind11-abi 4 hd8ed1ab_3 conda-forge pycosat 0.6.4 py310h5eee18b_0 pycparser 2.21 pyhd3eb1b0_0 pygam 0.9.0 pypi_0 pypi pyopenssl 23.0.0 py310h06a4308_0 pyparsing 3.0.9 pypi_0 pypi pysocks 1.7.1 py310h06a4308_0 python 3.10.12 hd12c33a_0_cpython conda-forge python-dateutil 2.8.2 pypi_0 pypi python-utils 3.7.0 pypi_0 pypi python_abi 3.10 3_cp310 conda-forge pytz 2023.3 pypi_0 pypi readline 8.2 h5eee18b_0 reproc 14.2.4 h0b41bf4_0 conda-forge reproc-cpp 14.2.4 hcb278e6_0 conda-forge requests 2.28.1 py310h06a4308_1 ruamel.yaml 0.17.21 py310h5eee18b_0 ruamel.yaml.clib 0.2.6 py310h5eee18b_1 scipy 1.11.1 pypi_0 pypi setuptools 65.6.3 py310h06a4308_0 six 1.16.0 pyhd3eb1b0_1 sqlite 3.41.1 h5eee18b_0 statsmodels 0.14.0 pypi_0 pypi tk 8.6.12 h1ccaba5_0 toolz 0.12.0 py310h06a4308_0 tqdm 4.65.0 py310h2f386ee_0 typing-extensions 4.7.1 pypi_0 pypi tzdata 2023.3 pypi_0 pypi urllib3 1.26.15 py310h06a4308_0 wheel 0.38.4 py310h06a4308_0 xz 5.2.10 h5eee18b_1 yaml-cpp 0.7.0 h27087fc_2 conda-forge zlib 1.2.13 hd590300_5 conda-forge zstandard 0.19.0 py310h5eee18b_0 zstd 1.5.2 hfc55251_7 conda-forge

Thank you

Sebastien-Raguideau commented 1 year ago

Yes I was thinking you could try again the command line: /mnt/strong/tools/apps/miniconda3.10/bin/mamba create -f /mnt/strong/tools/apps/STRONG/conda_env.yaml

But then if it fail it fail. You wrote in your first message that you tried different miniconda version and that the target directory is writable. Then it is harder than I would like :) I googled your error message, it is possibly an interaction of your os and conda? Or maybe not.

We can test if conda is possible at all, did you try installing some random software with conda/mamba? I would try installing anything in base, first with mamba (should fail) then with conda (hopefully doesn't). Then if conda works you can have it solve the env, which latest time I tried took 24h. But maybe they improved.

Final, tryhard solution is to create a singularity container (I should do that at some point) and run the strong install there.

jyee123 commented 1 year ago

$ /mnt/strong/tools/apps/miniconda3.10/bin/mamba create -f /mnt/strong/tools/apps/STRONG/conda_env.yaml usage: mamba create [-h] [--clone ENV] (-n ENVIRONMENT | -p PATH) [-c CHANNEL] [--use-local] [--override-channels] [--repodata-fn REPODATA_FNS] [--experimental {jlap,lock}] [--strict-channel-priority] [--no-channel-priority] [--no-deps | --only-deps] [--no-pin] [--copy] [-C] [-k] [--offline] [-d] [--json] [-q] [-v] [-y] [--download-only] [--show-channel-urls] [--file FILE] [--no-default-packages] [--solver {classic} | --experimental-solver {classic}] [--dev] [package_spec ...] mamba create: error: one of the arguments -n/--name -p/--prefix is required $ which mamba /mnt/strong/tools/apps/miniconda3.10/bin/mamba

"We can test if conda is possible at all". OK, I took the mamba piece out of the picture and used conda exclusively by performing "conda install" for the packages specified in conda_env.yaml.

That is, conda create STRONG conda activate STRONG conda install snakemake conda install metabat2 conda install biopython ... conda install xorg-libxt

One thing I did not do is specify the version number as specified in conda_env.yaml. What I noticed conda will stall if I try to specify anything other than the current version available. Also conda for some reason cannot install bam-readcount so I skipped it (conda install -c bioconda bam-readcount).

After I installed all the packages in the STRONG env, I attempted install_STRONG.sh script: $ ./install_STRONG.sh SPAdes install ongoing SPAdes install succesfully creating STRONG env /mnt/strong/tools/apps/miniconda3.10 Installing BayesPaths Installing DESMAN no issue found with install

=====> I guess the "no issue found with install" is a good sign so the next step is run your suggest test.

=====> test-config.yaml:

------ Samples ------

samples: ['sample'] # specify a list samples to use or '' to use all samples

------ Resources ------

threads : 8 # single task nb threads

------ Assembly parameters ------

data: /home/ubuntu/STRONG_Runs/Test # path to data folder

data: /mnt/strong/tools/apps/STRONG_Runs/Test # path to data folder

----- Annotation database -----

cog_database: /home/ubuntu/rpsblast_cog_db/Cog # COG database

cog_database: /mnt/strong/tools/apps/STRONG_database/rpsblast_cog_db/Cog # COG d atabase

----- Binner ------

binner: "concoct"

----- Binning parameters ------

concoct: contig_size: 1000

read_length: 150 assembly: assembler: spades k: [77] mem: 2000 threads: 24

----- BayesPaths parameters ------

bayespaths: nb_strains: 5 nmf_runs: 1 max_giter: 1 min_orf_number_to_merge_bins: 18 min_orf_number_to_run_a_bin: 10 percent_unitigs_shared: 0.1

----- DESMAN parameters ------

desman: execution: 1 nb_haplotypes: 10 nb_repeat: 5 min_cov: 1

----- Evaluation ------

evaluation: execution: 1

genomes: "/home/ubuntu/STRONG_Runs/Test/Eval" # path to reference genom

es genomes: "/mnt/strong/tools/apps/STRONG_Runs/Test/Eval" # path to reference genomes

=====> Received this error message: Job 7: Assembling with metaSPAdes Reason: Missing output files: assembly/spades/assembly.fasta, assembly/high_res/graph_pack.gfa

=====> Full output: ./bin/STRONG --config ./SnakeNest/test_config.yaml $SRPATH/TestResults --threads 8 --verbose

Step #1 - Assembly / Binning / COG Annotation Config file config.yaml is extended by additional config specified via the command line. Building DAG of jobs... Using shell: /usr/bin/bash Provided cores: 8 Rules claiming more threads will be scaled down. Job stats: job count min threads max threads

all 1 1 1 bed_orfs 1 1 1 bedtools_split_cov 8 1 1 bowtie_align 8 8 8 bowtie_index 1 8 8 cat_split_annotation 1 1 1 compute_avg_cov 1 1 1 concoct 1 8 8 copy_fasta 1 1 1 coverage 1 1 1 create_bin_folders 1 1 1 cut_contigs 1 1 1 extract_SCG_sequences 1 1 1 get_SCG_tables 1 1 1 initial_quantity_of_bins 1 1 1 merge_contigs 1 8 8 parse_cogs_annotation 1 1 1 prodigal 1 1 1 reads_yaml 1 1 1 refine 1 8 8 rpsblast 100 1 1 samtools_sort 8 8 8 simplify 1 8 8 spades 1 8 8 split_fasta 1 1 1 unitig_profiles 1 8 8 total 146 1 8

Resources before job selection: {'_cores': 8, '_nodes': 9223372036854775807} Ready jobs (2) Select jobs to execute... Welcome to the CBC MILP Solver Version: 2.10.10 Build Date: Apr 19 2023

command line - cbc /tmp/f18348a88825414bb730a38b3b0a5d33-pulp.mps max timeMode elapsed branch printingOptions all solution /tmp/f18348a88825414bb730a38b3b0a5d33-pulp.sol (default strategy 1) At line 2 NAME MODEL At line 3 ROWS At line 7 COLUMNS At line 18 RHS At line 21 BOUNDS At line 24 ENDATA Problem MODEL has 2 rows, 2 columns and 4 elements Coin0008I MODEL read with 0 errors Option for timeMode changed from cpu to elapsed Continuous objective value is 16 - 0.00 seconds Cgl0004I processed model has 1 rows, 2 columns (2 integer (2 of which binary)) and 2 elements Cutoff increment increased from 1e-05 to 1.9999 Cbc0038I Initial state - 0 integers unsatisfied sum - 0 Cbc0038I Solution found of 16 Cbc0038I Before mini branch and bound, 2 integers at bound fixed and 0 continuous Cbc0038I Mini branch and bound did not improve solution (0.00 seconds) Cbc0038I After 0.00 seconds - Feasibility pump exiting with objective of 16 - took 0.00 seconds Cbc0012I Integer solution of 16 found by feasibility pump after 0 iterations and 0 nodes (0.00 seconds) Cbc0001I Search completed - best objective 16, took 0 iterations and 0 nodes (0.00 seconds) Cbc0035I Maximum depth 0, 0 variables fixed on reduced cost Cuts at root node changed objective from 16 to 16 Probing was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds) Gomory was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds) Knapsack was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds) Clique was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds) MixedIntegerRounding2 was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds) FlowCover was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds) TwoMirCuts was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds) ZeroHalf was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)

Result - Optimal solution found

Objective value: 16.00000000 Enumerated nodes: 0 Total iterations: 0 Time (CPU seconds): 0.00 Time (Wallclock seconds): 0.00

Option for printingOptions changed from normal to all Total time (CPU seconds): 0.00 (Wallclock seconds): 0.00

Selected jobs (1) Resources after job selection: {'_cores': 0, '_nodes': 9223372036854775806}

[Sun Jul 16 14:09:14 2023] Job 7: Assembling with metaSPAdes Reason: Missing output files: assembly/spades/assembly.fasta, assembly/high_res/graph_pack.gfa

    /mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py --meta --only-assembler --save-gp -t 8 -k 77 -m 2000 -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R1.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R2.fq.gz -o assembly/spades/ &> assembly/spades.log
    cp assembly/spades//contigs.fasta assembly/spades/assembly.fasta
    cp assembly/spades//K77/saves/00_before_simplification/graph_pack* $(dirname assembly/high_res/graph_pack.gfa)

Full Traceback (most recent call last): File "/mnt/strong/tools/apps/miniconda3.10/envs/STRONG/lib/python3.10/site-packages/snakemake/executors/init.py", line 2561, in run_wrapper run( File "/mnt/strong/tools/apps/STRONG/SnakeNest/Assembly.snake", line 50, in __rule_spades File "/mnt/strong/tools/apps/miniconda3.10/envs/STRONG/lib/python3.10/site-packages/snakemake/shell.py", line 294, in new raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail; /mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py --meta --only-assembler --save-gp -t 8 -k 77 -m 2000 -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R1.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R2.fq.gz -o assembly/spades/ &> assembly/spades.log cp assembly/spades//contigs.fasta assembly/spades/assembly.fasta cp assembly/spades//K77/saves/00_before_simplification/graph_pack* $(dirname assembly/high_res/graph_pack.gfa)' returned non-zero exit status 1.

[Sun Jul 16 14:09:14 2023] Error in rule spades: jobid: 7 input: /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R1.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R1.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R1.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R1.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R1.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R1.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R1.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R1.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R2.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R2.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R2.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R2.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R2.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R2.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R2.fq.gz, /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R2.fq.gz output: assembly/spades/assembly.fasta, assembly/high_res/graph_pack.gfa log: assembly/spades.log (check log file(s) for error details) shell:

    /mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py --meta --only-assembler --save-gp -t 8 -k 77 -m 2000 -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R1.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R2.fq.gz -o assembly/spades/ &> assembly/spades.log
    cp assembly/spades//contigs.fasta assembly/spades/assembly.fasta
    cp assembly/spades//K77/saves/00_before_simplification/graph_pack* $(dirname assembly/high_res/graph_pack.gfa)

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Full Traceback (most recent call last): File "/mnt/strong/tools/apps/miniconda3.10/envs/STRONG/lib/python3.10/site-packages/snakemake/executors/init.py", line 2561, in run_wrapper run( File "/mnt/strong/tools/apps/STRONG/SnakeNest/Assembly.snake", line 50, in __rule_spades File "/mnt/strong/tools/apps/miniconda3.10/envs/STRONG/lib/python3.10/site-packages/snakemake/shell.py", line 294, in new raise sp.CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -euo pipefail; /mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py --meta --only-assembler --save-gp -t 8 -k 77 -m 2000 -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R1.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R2.fq.gz -o assembly/spades/ &> assembly/spades.log cp assembly/spades//contigs.fasta assembly/spades/assembly.fasta cp assembly/spades//K77/saves/00_before_simplification/graph_pack* $(dirname assembly/high_res/graph_pack.gfa)' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/mnt/strong/tools/apps/miniconda3.10/envs/STRONG/lib/python3.10/site-packages/snakemake/executors/init.py", line 656, in _callback raise ex File "/mnt/strong/tools/apps/miniconda3.10/envs/STRONG/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) File "/mnt/strong/tools/apps/miniconda3.10/envs/STRONG/lib/python3.10/site-packages/snakemake/executors/init.py", line 642, in cached_or_run run_func(args) File "/mnt/strong/tools/apps/miniconda3.10/envs/STRONG/lib/python3.10/site-packages/snakemake/executors/init.py", line 2597, in run_wrapper raise RuleException( snakemake.exceptions.RuleException: CalledProcessError in file /mnt/strong/tools/apps/STRONG/SnakeNest/Assembly.snake, line 22: Command 'set -euo pipefail; /mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py --meta --only-assembler --save-gp -t 8 -k 77 -m 2000 -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R1.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R2.fq.gz -o assembly/spades/ &> assembly/spades.log cp assembly/spades//contigs.fasta assembly/spades/assembly.fasta cp assembly/spades//K77/saves/00_before_simplification/graph_pack* $(dirname assembly/high_res/graph_pack.gfa)' returned non-zero exit status 1. File "/mnt/strong/tools/apps/STRONG/SnakeNest/Assembly.snake", line 22, in __rule_spades

RuleException: CalledProcessError in file /mnt/strong/tools/apps/STRONG/SnakeNest/Assembly.snake, line 22: Command 'set -euo pipefail; /mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py --meta --only-assembler --save-gp -t 8 -k 77 -m 2000 -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R1.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R2.fq.gz -o assembly/spades/ &> assembly/spades.log cp assembly/spades//contigs.fasta assembly/spades/assembly.fasta cp assembly/spades//K77/saves/00_before_simplification/graph_pack* $(dirname assembly/high_res/graph_pack.gfa)' returned non-zero exit status 1. File "/mnt/strong/tools/apps/STRONG/SnakeNest/Assembly.snake", line 22, in __rule_spades File "/mnt/strong/tools/apps/miniconda3.10/envs/STRONG/lib/python3.10/concurrent/futures/thread.py", line 58, in run Resources before job selection: {'_cores': 8, '_nodes': 9223372036854775807} Ready jobs (1) Select jobs to execute... Using greedy selector because only single job has to be scheduled. Selected jobs (1) Resources after job selection: {'_cores': 7, '_nodes': 9223372036854775806}

[Sun Jul 16 14:09:14 2023] rule reads_yaml: output: samples.yaml jobid: 33 reason: Missing output files: samples.yaml resources: tmpdir=/tmp

Config file config.yaml is extended by additional config specified via the command line. Building DAG of jobs... Using shell: /usr/bin/bash Provided cores: 8 Rules claiming more threads will be scaled down. Select jobs to execute... [Sun Jul 16 14:09:15 2023] Finished job 33. 1 of 146 steps (1%) done Exiting because a job execution failed. Look above for error message Complete log: .snakemake/log/2023-07-16T140914.151182.snakemake.log unlocking removing lock removing lock removed all locks Traceback (most recent call last): File "/mnt/strong/tools/apps/STRONG/./bin/STRONG", line 94, in call_snake(["--snakefile", "SnakeNest/SCogSubGraph.snake"]) File "/mnt/strong/tools/apps/STRONG/./bin/STRONG", line 83, in call_snake subprocess.check_call(base_params + extra_params, stdout=sys.stdout, stderr=sys.stderr) File "/mnt/strong/tools/apps/miniconda3.10/envs/STRONG/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['snakemake', '--directory', '/mnt/strong/tools/apps/STRONG_Runs/TestResults', '--cores', '8', '--config', 'LOCAL_DIR=/mnt/strong/tools/apps/STRONG', '--configfile=/mnt/strong/tools/apps/STRONG_Runs/TestResults/config.yaml', '--latency-wait', '120', '-k', '-p', '-r', '--verbose', '--snakefile', 'SnakeNest/SCogSubGraph.snake']' returned non-zero exit status 1.

Sebastien-Raguideau commented 1 year ago

Hi,

Happy you managed to install most things. I don't remember but I think bam-readcount is needed mostly for Desman.

If you don't mind, the log would be better as a joined file, it's a bit hard to read here.

Anyway, there is an issue with spades. Can you join the log file which is at: assembly/spades.log. The install script checks that one of the utils has been compiled, this should be sufficient check but maybe there is still an issue?

jyee123 commented 1 year ago

Hello, part 1 of 3. install.log from the STRONG directory (3678 lines). install.log

jyee123 commented 1 year ago

Hello, part 2 of 3. /mnt/strong/tools/apps/STRONG_Runs/TestResults/assembly/spades.log

$ cat /mnt/strong/tools/apps/STRONG_Runs/TestResults/assembly/spades.log Command line: /mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py --meta --only-assembler --save-gp -t 8 -k 77 -m 2000 -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R1.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R2.fq.gz -o /mnt/strong/tools/apps/STRONG_Runs/TestResults/assembly/spades

System information: SPAdes version: 3.14.0 Python version: 3.10.12 OS: Linux-4.18.0-425.3.1.el8.x86_64-x86_64-with-glibc2.28

Output dir: /mnt/strong/tools/apps/STRONG_Runs/TestResults/assembly/spades Mode: ONLY assembling (without read error correction) Debug mode is turned OFF

Dataset parameters: Metagenomic mode Reads: Traceback (most recent call last): File "/mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py", line 639, in main(sys.argv) File "/mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py", line 579, in main print_params(log, log_filename, command_line, args, cfg) File "/mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py", line 322, in print_params print_used_values(cfg, log) File "/mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py", line 112, in print_used_values dataset_data = pyyaml.load(open(cfg["dataset"].yaml_filename)) File "/mnt/strong/tools/apps/STRONG/SPAdes/assembler/share/spades/pyyaml3/init.py", line 72, in load return loader.get_single_data() File "/mnt/strong/tools/apps/STRONG/SPAdes/assembler/share/spades/pyyaml3/constructor.py", line 37, in get_single_data return self.construct_document(node) File "/mnt/strong/tools/apps/STRONG/SPAdes/assembler/share/spades/pyyaml3/constructor.py", line 46, in construct_document for dummy in generator: File "/mnt/strong/tools/apps/STRONG/SPAdes/assembler/share/spades/pyyaml3/constructor.py", line 398, in construct_yaml_map value = self.construct_mapping(node) File "/mnt/strong/tools/apps/STRONG/SPAdes/assembler/share/spades/pyyaml3/constructor.py", line 204, in construct_mapping return super().construct_mapping(node, deep=deep) File "/mnt/strong/tools/apps/STRONG/SPAdes/assembler/share/spades/pyyaml3/constructor.py", line 126, in construct_mapping if not isinstance(key, collections.Hashable): AttributeError: module 'collections' has no attribute 'Hashable'

jyee123 commented 1 year ago

Hello, part 3 of 3. /mnt/strong/tools/apps/STRONG_Runs/TestResults/assembly/spades/spades.log

$ cat /mnt/strong/tools/apps/STRONG_Runs/TestResults/assembly/spades/spades.log Command line: /mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py --meta --only-assembler --save-gp -t 8 -k 77 -m 2000 -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R1.fq.gz -1 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R1.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample1/sample1_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample2/sample2_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample3/sample3_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample4/sample4_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample5/sample5_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample6/sample6_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample7/sample7_R2.fq.gz -2 /mnt/strong/tools/apps/STRONG_Runs/Test/sample8/sample8_R2.fq.gz -o /mnt/strong/tools/apps/STRONG_Runs/TestResults/assembly/spades

System information: SPAdes version: 3.14.0 Python version: 3.10.12 OS: Linux-4.18.0-425.3.1.el8.x86_64-x86_64-with-glibc2.28

Output dir: /mnt/strong/tools/apps/STRONG_Runs/TestResults/assembly/spades Mode: ONLY assembling (without read error correction) Debug mode is turned OFF

Dataset parameters: Metagenomic mode Reads:

Sebastien-Raguideau commented 1 year ago

Hi I googled that error message, it seems to be an issue with spades version. Can you update it to 3.15.4 using: https://cab.spbu.ru/files/release3.15.4/manual.html#sec2

jyee123 commented 1 year ago

Hello, thank you for all the help so far. I downloaded the precompiled Linux version and still the same error message when I run the test job: [Tue Jul 18 09:24:12 2023] Job 7: Assembling with metaSPAdes Reason: Missing output files: assembly/high_res/graph_pack.gfa, assembly/spades/assembly.fasta

Here are my steps, $ cd /mnt/strong/tools/apps/STRONG $ mv SPAdes SPAdes-original $ wget --no-check-certificate http://cab.spbu.ru/files/release3.15.4/SPAdes-3.15.4-Linux.tar.gz $ tar -xzf SPAdes-3.15.4-Linux.tar.gz $ ln -s SPAdes-3.15.4-Linux SPAdes $ ls SPAdes/bin/ cds-mapping-stats rnaviralspades.py spades_init.py cds-subgraphs spades-bwa spades-ionhammer coronaspades.py spades-convert-bin-to-fasta spades-kmercount mag-improve spades-core spades-kmer-estimating metaplasmidspades.py spades-corrector-core spades.py metaspades.py spades-gbuilder spades-read-filter metaviralspades.py spades-gmapper spades-truseq-scfcorrection plasmidspades.py spades-gsimplifier spaligner rnaspades.py spades-hammer truspades.py

check_on_dependencies.py and new SPAdes: ------------------------------------------------------------------------------ Following issues where found : SPAdes is not compiled ------------------------------------------------------------------------------

check_on_dependencies.py and original SPAdes: no issue found with install

Sebastien-Raguideau commented 1 year ago

Hi, Hum, that in itself is not an error message but just information on what task is being run (assembly with spade, since output have not been generated previously). When I run it on terminal, i see these message in green color. If it fail due to any sort of error, you would have it in red? Also, for spade assembly you the logs would be more informative, the one at: assembly/spades.log.

jyee123 commented 1 year ago

Hello, I think you want see the test run output in color? Please have a look at this pdf link. SPAdes-test01.pdf

Here's the output of spades.log: $ cat /mnt/strong/tools/apps/STRONG_Runs/TestResults/assembly/spades.log /usr/bin/bash: line 1: /mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py: No such file or directory

I'm running the precompiled Linux binary of SPAdes downloaded from the site you suggested. The precompiled version does not contain the assembler directory.

Thank you

Sebastien-Raguideau commented 1 year ago

Hello, Thanks for the pdf, that's neat. How did you even do that? Anyway, I'm sorry but I'm going to be limited in how I can help you. Unless I'm wrong, you have an issue of installation where you did install that new version of spades but not exactly where STRONG is expecting it. Strong is expecting it in the STRONG repos. You need to either reinstall spades in the correct folder or just do a symlink and replace /mnt/strong/tools/apps/STRONG/SPAdes/assembler/bin/spades.py by the new install you have, which, if you followed the instructions to the letter should be at: /bin/spades.py Best, Seb

jyee123 commented 1 year ago

Hello, I used putty to copy and paste the text into Word which retained the color.

For the past two weeks, on and off I've been trying to install STRONG natively using the instructions on this github page (Plan A). When this didn't work I initiated this thread (Plan B). I tried Plan A again today and the mamba portion of install_STRONG.sh worked and it returned "no issue found with install". I even executed your suggested test job which ran for about 40 minutes and all I saw was white, blue, yellow and green text. So I think it all worked. Something with the mamba backend must be the culprit as my environment didn't change and I was consistent on how I perform the installation. So at least for now I'm good and have a good working version of STRONG.

Sorry, this may have been a false alarm.

Thanks for your time on helping me troubleshoot.

Sebastien-Raguideau commented 1 year ago

Happy it worked in the end :)