PacificBiosciences / pbbioconda

PacBio Secondary Analysis Tools on Bioconda. Contains list of PacBio packages available via conda.
BSD 3-Clause Clear License
251 stars 45 forks source link

IPA error #479

Closed MartonaAlvarez closed 1 year ago

MartonaAlvarez commented 2 years ago

Operating system Which operating system and version are you using? I am working on a server with:

NAME="Ubuntu"
VERSION="14.04.5 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.5 LTS"
VERSION_ID="14.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"

Package name Which package / tool is causing the problem? Which version are you using, use tool --version. Have you updated to the latest version conda update package? Have you updated the complete env by running conda update --all? Have you ensured that your channel priorities are set up according to the bioconda recommendations at https://bioconda.github.io/#set-up-channels?

I am using IPA installed with conda with all last versions.

Conda environment What is the result of conda list? (Try to paste that between triple backticks.)

# packages in environment at /home_new/jordi/soft/miniconda3/envs/ipa:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
amply                     0.1.4                      py_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
attrs                     21.4.0             pyhd8ed1ab_0    conda-forge
brotli                    1.0.9                h7f98852_6    conda-forge
brotli-bin                1.0.9                h7f98852_6    conda-forge
brotlipy                  0.7.0           py310h6acc77f_1003    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2021.10.8            ha878542_0    conda-forge
certifi                   2021.10.8       py310hff52083_1    conda-forge
cffi                      1.15.0          py310h0fdd8cc_0    conda-forge
charset-normalizer        2.0.12             pyhd8ed1ab_0    conda-forge
coincbc                   2.10.5               hcee13e7_1    conda-forge
configargparse            1.5.3              pyhd8ed1ab_0    conda-forge
connection_pool           0.0.3              pyhd3deb0d_0    conda-forge
cryptography              36.0.1          py310h685ca39_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
datrie                    0.8.2           py310h6acc77f_3    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
docutils                  0.18.1          py310hff52083_0    conda-forge
filelock                  3.6.0              pyhd8ed1ab_0    conda-forge
fonttools                 4.29.1          py310h6acc77f_0    conda-forge
freetype                  2.10.4               h0708190_1    conda-forge
giflib                    5.2.1                h36c2ea0_2    conda-forge
gitdb                     4.0.9              pyhd8ed1ab_0    conda-forge
gitpython                 3.1.27             pyhd8ed1ab_0    conda-forge
htslib                    1.14                 h9753748_2    bioconda
idna                      3.3                pyhd8ed1ab_0    conda-forge
importlib-metadata        4.11.2          py310hff52083_0    conda-forge
importlib_resources       5.4.0              pyhd8ed1ab_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
jbig                      2.1               h7f98852_2003    conda-forge
jinja2                    3.0.3              pyhd8ed1ab_0    conda-forge
jpeg                      9e                   h7f98852_0    conda-forge
jsonschema                4.4.0              pyhd8ed1ab_0    conda-forge
jupyter_core              4.9.2           py310hff52083_0    conda-forge
k8                        0.2.5                hd03093a_2    bioconda
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.3.2           py310h91b1402_1    conda-forge
krb5                      1.19.2               h3790be6_4    conda-forge
lcms2                     2.12                 hddcbb42_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
lerc                      3.0                  h9c3ff4c_0    conda-forge
libblas                   3.9.0           13_linux64_openblas    conda-forge
libbrotlicommon           1.0.9                h7f98852_6    conda-forge
libbrotlidec              1.0.9                h7f98852_6    conda-forge
libbrotlienc              1.0.9                h7f98852_6    conda-forge
libcblas                  3.9.0           13_linux64_openblas    conda-forge
libcurl                   7.81.0               h2574ce0_0    conda-forge
libdeflate                1.10                 h7f98852_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 11.2.0              h1d223b6_13    conda-forge
libgfortran-ng            11.2.0              h69a702a_13    conda-forge
libgfortran5              11.2.0              h5c6108e_13    conda-forge
libgomp                   11.2.0              h1d223b6_13    conda-forge
liblapack                 3.9.0           13_linux64_openblas    conda-forge
libnghttp2                1.47.0               h727a467_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libopenblas               0.3.18          pthreads_h8fe5266_0    conda-forge
libpng                    1.6.37               h21135ba_2    conda-forge
libssh2                   1.10.0               ha56f1ee_2    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_13    conda-forge
libtiff                   4.3.0                h542a066_3    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libwebp                   1.2.2                h3452ae3_0    conda-forge
libwebp-base              1.2.2                h7f98852_1    conda-forge
libxcb                    1.13              h7f98852_1004    conda-forge
libzlib                   1.2.11            h36c2ea0_1013    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
markupsafe                2.1.0           py310h5764c6d_1    conda-forge
matplotlib-base           3.5.1           py310h23f4a51_0    conda-forge
minimap2                  2.24                 h7132678_1    bioconda
munkres                   1.0.7                      py_1    bioconda
nbformat                  5.1.3              pyhd8ed1ab_0    conda-forge
ncurses                   6.3                  h9c3ff4c_0    conda-forge
networkx                  2.7.1              pyhd8ed1ab_0    conda-forge
numpy                     1.22.2          py310h454958d_0    conda-forge
openjpeg                  2.4.0                hb52868f_1    conda-forge
openssl                   1.1.1l               h7f98852_0    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pandas                    1.4.1           py310hb5077e9_0    conda-forge
pb-falconc                1.13.1               h3279499_2    bioconda
pbipa                     1.5.0                h259a17a_2    bioconda
pcre                      8.45                 h9c3ff4c_0    conda-forge
pillow                    9.0.1           py310he619898_2    conda-forge
pip                       22.0.4             pyhd8ed1ab_0    conda-forge
plac                      1.3.4              pyhd8ed1ab_0    conda-forge
psutil                    5.9.0           py310h6acc77f_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pulp                      2.6.0           py310hff52083_0    conda-forge
py                        1.11.0             pyh6c4a22f_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyopenssl                 22.0.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.7              pyhd8ed1ab_0    conda-forge
pyrsistent                0.18.1          py310h6acc77f_0    conda-forge
pysocks                   1.7.1           py310hff52083_4    conda-forge
python                    3.10.2          h85951f9_3_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.10                    2_cp310    conda-forge
pytz                      2021.3             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0             py310h6acc77f_3    conda-forge
racon                     1.4.20               hd03093a_2    bioconda
ratelimiter               1.2.0                   py_1002    conda-forge
readline                  8.1                  h46c0cb4_0    conda-forge
requests                  2.27.1             pyhd8ed1ab_0    conda-forge
retry                     0.9.2                      py_0    conda-forge
samtools                  1.15                 h3843a85_0    bioconda
scipy                     1.8.0           py310hea5193d_1    conda-forge
setuptools                60.9.3          py310hff52083_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
smart_open                5.2.1              pyhd8ed1ab_0    conda-forge
smmap                     3.0.5              pyh44b312d_0    conda-forge
snakemake-minimal         7.1.0              pyhdfd78af_0    bioconda
sqlite                    3.37.0               h9cd32fc_0    conda-forge
stopit                    1.1.2                      py_0    conda-forge
tabulate                  0.8.9              pyhd8ed1ab_0    conda-forge
time                      1.8                  h516909a_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
toposort                  1.7                pyhd8ed1ab_0    conda-forge
traitlets                 5.1.1              pyhd8ed1ab_0    conda-forge
typing_extensions         4.1.1              pyha770c72_0    conda-forge
tzdata                    2021e                he74cb21_0    conda-forge
unicodedata2              14.0.0          py310h6acc77f_0    conda-forge
urllib3                   1.26.8             pyhd8ed1ab_1    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
wrapt                     1.13.3          py310h6acc77f_1    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yte                       1.2.0           py310hff52083_0    conda-forge
zipp                      3.7.0              pyhd8ed1ab_1    conda-forge
zlib                      1.2.11            h36c2ea0_1013    conda-forge
zstd                      1.5.2                ha95c52a_0    conda-forge

Describe the bug A clear and concise description of what the bug is. I have run the IPA program to assemble a genome with HiFi reads, but it always stops at the same step, giving the same message. This is the code: ipa local -i bothrioHiFi_reads.fastq --njobs 2 --nthreads 25 --genome-size 4943861964 --coverage 18 I have tried to restart it, with the --resume flag, as I have seen in some posts, but it keeps failing. I have also tried to run snakemake alone, with the command line indicated in the log file, but it doesn't work either. miniconda3/envs/ipa/bin/python3 -m snakemake -j 2 -d RUN -p -s miniconda3/envs/ipa/etc/ipa.snakefile --configfile RUN/config.yaml --reason I don't know what could be wrong.

Error message Paste the error message / stack.

Error in rule build_db:
    jobid: 5
    output: 02-build_db/reads.seqdb, 02-build_db/reads.seeddb, 02-build_db/reads.seqdb.0.seq, 02-build_db/reads.seeddb.0.seeds, 02-build_db/input.fofn
    shell:

        wd=$(dirname 02-build_db/reads.seqdb)
        mkdir -p $wd
        cd $wd
        rel=..

        input_reads_fn="/mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/input.fofn"         params_db_prefix="reads"         params_config_sh_fn="$rel/01-generate_config/generated.config"         params_num_threads=25         params_log_level="INFO"         params_tmp_dir="/tmp"             time ipa2-task build_db

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job build_db since they might be corrupted:
02-build_db/reads.seqdb.0.seq, 02-build_db/input.fofn
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/.snakemake/log/2022-03-10T175133.540711.snakemake.log

To Reproduce Steps to reproduce the behavior. Providing a minimal test dataset on which we can reproduce the behavior will generally lead to quicker turnaround time!

Expected behavior A clear and concise description of what you expected to happen.

isovic commented 2 years ago

Hi @MartonaAlvarez,

Can you please post the entire stderr of the ipa local output (not just this last chunk)? There seems to be something wrong in the DB construction stage.

Can you also please verify that bothrioHiFi_reads.fastq is a valid FASTQ file?

Best regards, Ivan.

MartonaAlvarez commented 2 years ago

Hi @isovic , thanks for your response. Here is the output complete. Regarding the fastq file, yes, it is valid.

INFO: /home_new/jordi/soft/miniconda3/envs/ipa/bin/ipa local -i bothrioHiFi_reads.fastq --njobs 2 --nthreads 25 --genome-size 4943861964 --coverage 18 --resume
INFO: ipa.py ipa (wrapper) version=1.5.0 ... Checking dependencies ...
INFO: Dependencies
/home_new/jordi/soft/miniconda3/envs/ipa/bin/python3
/home_new/jordi/soft/miniconda3/envs/ipa/bin/ipa2-task
/home_new/jordi/soft/miniconda3/envs/ipa/bin/falconc
/home_new/jordi/soft/miniconda3/envs/ipa/bin/minimap2
/home_new/jordi/soft/miniconda3/envs/ipa/bin/nighthawk
/home_new/jordi/soft/miniconda3/envs/ipa/bin/pancake
/home_new/jordi/soft/miniconda3/envs/ipa/bin/pblayout
/home_new/jordi/soft/miniconda3/envs/ipa/bin/racon
/home_new/jordi/soft/miniconda3/envs/ipa/bin/samtools
/home_new/jordi/soft/miniconda3/envs/ipa/bin/ipa_purge_dups
/home_new/jordi/soft/miniconda3/envs/ipa/bin/ipa_purge_dups_split_fa
snakemake version=7.1.0
ipa2-task 1.5.0 (commit c875fce13bdacbafc2f4f750c6438f4453e1354d)
 Machine name: 'Linux'
Copyright (C) 2004-2021     Pacific Biosciences of California, Inc.
This program comes with ABSOLUTELY NO WARRANTY; it is intended for
Research Use Only and not for use in diagnostic procedures.

falconc version=1.13.1+git.f9d1b5651e891efe379bd9727a0fa0931b875d7b, nim-version=1.5.1
minimap2 version=2.24-r1122
Nighthawk 0.1.0 (commit SL-release-10.1.0-7-gbe5dfb1*)
pancake 1.3.0 (commit SEQII-release-10.1.0-432-gf2693fd*)
pblayout 1.0.0 (commit SL-release-10.1.0-152-g66936d1*)
racon version=v1.4.20
samtools 1.15
Using htslib 1.14
ipa_purge_dups Version: 1.2.5

To run this yourself:
/home_new/jordi/soft/miniconda3/envs/ipa/bin/python3 -m snakemake -j 2 -d RUN -p -s /home_new/jordi/soft/miniconda3/envs/ipa/etc/ipa.snakefile --configfile RUN/config.yaml --reason

Starting snakemake ...
CWD:/mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN
NPROC:25
NPROC_SERIAL:25
config:{'advanced_options': '', 'coverage': 18, 'genome_size': 4943861964, 'm4filt_high_copy_sample_rate': 1.0, 'max_nchunks': 40, 'nproc': 25, 'phase_run': 1, 'polish_run': 1, 'purge_dups_calcuts': '', 'purge_dups_run': 1, 'reads_fn': 'RUN/input.fofn', 'tmp_dir': '/tmp'}
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 2
Rules claiming more threads will be scaled down.
Job stats:
job                       count    min threads    max threads
----------------------  -------  -------------  -------------
assemble                      1              1              1
build_db                      1              1              1
final                         1              1              1
finish                        1              1              1
ovl_asym_merge                1              1              1
ovl_asym_prepare              1              1              1
ovl_filter                    1              1              1
phasing_merge                 1              1              1
phasing_prepare               1              1              1
polish_merge                  1              1              1
polish_prepare                1              1              1
purge_dups_map_merge          1              1              1
purge_dups_map_prepare        1              1              1
purge_dups_paf                1              1              1
separate_p_from_a             1              1              1
total                        15              1              1

Select jobs to execute...

[Mon Mar 14 10:15:52 2022]
rule build_db:
    input: /mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/input.fofn, 01-generate_config/generated.config
    output: 02-build_db/reads.seqdb, 02-build_db/reads.seeddb, 02-build_db/reads.seqdb.0.seq, 02-build_db/reads.seeddb.0.seeds, 02-build_db/input.fofn
    jobid: 5
    reason: Missing output files: 02-build_db/reads.seqdb
    resources: tmpdir=/tmp

        wd=$(dirname 02-build_db/reads.seqdb)
        mkdir -p $wd
        cd $wd
        rel=..

        input_reads_fn="/mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/input.fofn"         params_db_prefix="reads"         params_config_sh_fn="$rel/01-generate_config/generated.config"         params_num_threads=25         params_log_level="INFO"         params_tmp_dir="/tmp"             time ipa2-task build_db

/mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN /mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/02-build_db
/mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/02-build_db
/mnt/iscsi01-copy/userdir/Bothrioplana/ipa/bothrioHiFi_reads.fastq
>|> 20220314 11:57:02.418 -|- INFO -|- Runner -|- 0x7f35392e5780|| -|- Loading the input DBs.
Command exited with non-zero status 139
5727.91user 404.48system 1:42:15elapsed 99%CPU (0avgtext+0avgdata 2127640maxresident)k
0inputs+46524440outputs (0major+960864minor)pagefaults 0swaps
[Mon Mar 14 11:58:07 2022]
Error in rule build_db:
    jobid: 5
    output: 02-build_db/reads.seqdb, 02-build_db/reads.seeddb, 02-build_db/reads.seqdb.0.seq, 02-build_db/reads.seeddb.0.seeds, 02-build_db/input.fofn
    shell:

        wd=$(dirname 02-build_db/reads.seqdb)
        mkdir -p $wd
        cd $wd
        rel=..

        input_reads_fn="/mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/input.fofn"         params_db_prefix="reads"         params_config_sh_fn="$rel/01-generate_config/generated.config"         params_num_threads=25         params_log_level="INFO"         params_tmp_dir="/tmp"             time ipa2-task build_db

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job build_db since they might be corrupted:
02-build_db/reads.seqdb.0.seq, 02-build_db/input.fofn
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/.snakemake/log/2022-03-14T101551.012769.snakemake.log
ERROR: Command '/home_new/jordi/soft/miniconda3/envs/ipa/bin/python3 -m snakemake -j 2 -d RUN -p -s /home_new/jordi/soft/miniconda3/envs/ipa/etc/ipa.snakefile --configfile RUN/config.yaml --reason' returned non-zero exit status 1.
Exiting.

Thanks,

Marta

MartonaAlvarez commented 2 years ago

Hi,

I solved the first error and managed to get it running, however, it stops at Snakemake step 5 and I don't understand why. This is the message from snakemake:

Error in rule ovl_asym_merge:
    jobid: 13
    output: 05-ovl_asym_merge/ovl.merged.m4, 05-ovl_asym_merge/ovl.nonlocal.m4
    shell:

        wd=$(dirname 05-ovl_asym_merge/ovl.merged.m4)
        mkdir -p $wd
        cd $wd
        rel=..

        # We must change rel-path names when we chdir.
        for fn in 04-ovl_asym_run/18/ovl.sorted.m4 04-ovl_asym_run/32/ovl.sorted.m4 04-ovl_asym_run/12/ovl.sorted.m4 04-ovl_asym_run/16/ovl.sorted.m4 04-ovl_asym_run/30/ovl.sorted.m4 04-ovl_asym_run/29/ovl.sorted.m4 04-ovl_asym_run/23/ovl.sorted.m4 04-ovl_asym_run/31/ovl.sorted.m4 04-ovl_asym_run/1/ovl.sorted.m4 04-ovl_asym_run/5/ovl.sorted.m4 04-ovl_asym_run/25/ovl.sorted.m4 04-ovl_asym_run/0/ovl.sorted.m4 04-ovl_asym_run/4/ovl.sorted.m4 04-ovl_asym_run/37/ovl.sorted.m4 04-ovl_asym_run/38/ovl.sorted.m4 04-ovl_asym_run/6/ovl.sorted.m4 04-ovl_asym_run/2/ovl.sorted.m4 04-ovl_asym_run/9/ovl.sorted.m4 04-ovl_asym_run/33/ovl.sorted.m4 04-ovl_asym_run/10/ovl.sorted.m4 04-ovl_asym_run/17/ovl.sorted.m4 04-ovl_asym_run/14/ovl.sorted.m4 04-ovl_asym_run/3/ovl.sorted.m4 04-ovl_asym_run/24/ovl.sorted.m4 04-ovl_asym_run/27/ovl.sorted.m4 04-ovl_asym_run/20/ovl.sorted.m4 04-ovl_asym_run/35/ovl.sorted.m4 04-ovl_asym_run/13/ovl.sorted.m4 04-ovl_asym_run/19/ovl.sorted.m4 04-ovl_asym_run/26/ovl.sorted.m4 04-ovl_asym_run/34/ovl.sorted.m4 04-ovl_asym_run/39/ovl.sorted.m4 04-ovl_asym_run/36/ovl.sorted.m4 04-ovl_asym_run/8/ovl.sorted.m4 04-ovl_asym_run/28/ovl.sorted.m4 04-ovl_asym_run/11/ovl.sorted.m4 04-ovl_asym_run/22/ovl.sorted.m4 04-ovl_asym_run/15/ovl.sorted.m4 04-ovl_asym_run/21/ovl.sorted.m4 04-ovl_asym_run/7/ovl.sorted.m4; do
            echo $rel/$fn
        done >| ./sorted.fofn

        input_fofn=./sorted.fofn         params_num_threads="25"         params_db_prefix=reads         params_config_sh_fn="$rel/01-generate_config/generated.config"         params_log_level="INFO"         params_tmp_dir="/tmp"             time ipa2-task ovl_asym_merge

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job ovl_asym_merge since they might be corrupted:
05-ovl_asym_merge/ovl.merged.m4
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/.snakemake/log/2022-03-15T092702.530410.snakemake.log

Any help will be welcome. I have tried rerunning with the --resume flag and tried to run the snakemake commands only. But always get the same error.

Thanks, Marta

isovic commented 2 years ago

Hi Marta,

Can you please describe how you solved the first issue, for future reference?

I solved the first error and managed to get it running, however, it stops at Snakemake step 5 and I don't understand why. This is the message from snakemake:

Is there anything else written to stderr which might provide more info? Snakemake output, like the one you copy/pasted, is good at telling us that something went wrong, but not exactly what happened. The actual error messages, if there are any, are usually located just above/below this for an ipa local run, or in the cluster log file for that stage (located in RUN/qsub_log/).

This stage is actually very simple - all it does, literally, is concatenating all overlap files, sorting them and removing local alignments; all of which is done using standard Bash tools.

Is it possible that your disk is full?

Or that the temporary directory is not available on your system or is full (/tmp)? In this case, try running IPA with this additional parameter: `--tmp-dir "./"

Best regards, Ivan.

MartonaAlvarez commented 2 years ago

Hi Ivan,

thanks for your reply. For getting it running I splitted the input files to the original files I had from the demultiplexing step. It worked well.

It seems that the error this time was the one about the tmp dir, because I changed it to --tmp-dir ${PWD} and it worked.

The RUN/qsub_log/ directory is empty, so I only can give you information about the snakemake log file, I'm afraid.

Now the error is in step 9 (assembly). I've tried to rerun, but it doesn't work this time.

´´´ Finished job 11. 44 of 53 steps (83%) done Select jobs to execute...

[Sun Mar 27 16:45:24 2022] rule assemble: input: /mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/input.fofn, 02-build_db/reads.seqdb, 09-ovl_filter/ovl.final.m4, 08-phasing_merge/ovl.phased.m4, 01-generate_config/generated.config output: 10-assemble/p_ctg.fasta, 10-assemble/a_ctg.fasta, 10-assemble/p_ctg_tiling_path, 10-assemble/a_ctg_tiling_path, 10-assemble/p_ctg.fasta.fai, 10-assemble/a_ctg.fasta.fai, 10-assemble/read_to_contig.paf, 10-assemble/circular_contigs.csv jobid: 10 reason: Missing output files: 10-assemble/p_ctg.fasta, 10-assemble/a_ctg.fasta, 10-assemble/a_ctg.fasta.fai, 10-assemble/read_to_contig.paf, 10-assemble/p_ctg.fasta.fai; Input files updated by another job: 09-ovl_filter/ovl.final.m4, 08-phasing_merge/ovl.phased.m4 resources: tmpdir=/tmp

    wd=$(dirname 10-assemble/p_ctg.fasta)
    mkdir -p $wd
    cd $wd
    rel=..

    output_read_to_contig=$(basename 10-assemble/read_to_contig.paf)         input_seqdb="$rel/02-build_db/reads.seqdb"         input_m4="$rel/09-ovl_filter/ovl.final.m4"         input_m4_phasing_merge="$rel/08-phasing_merge/ovl.phased.m4"         input_reads="/mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/input.fofn"         params_ctg_prefix=ctg         params_num_threads=25         params_config_sh_fn="$rel/01-generate_config/generated.config"         params_log_level="INFO"         params_tmp_dir="/mnt/iscsi01-copy/userdir/Bothrioplana/ipa"             time ipa2-task assemble

[Sun Mar 27 16:45:24 2022] Error in rule assemble: jobid: 10 output: 10-assemble/p_ctg.fasta, 10-assemble/a_ctg.fasta, 10-assemble/p_ctg_tiling_path, 10-assemble/a_ctg_tiling_path, 10-assemble/p_ctg.fasta.fai, 10-assemble/a_ctg.fasta.fai, 10-assemble/read_to_contig.paf, 10-assemble/circular_contigs.csv shell:

    wd=$(dirname 10-assemble/p_ctg.fasta)
    mkdir -p $wd
    cd $wd
    rel=..

    output_read_to_contig=$(basename 10-assemble/read_to_contig.paf)         input_seqdb="$rel/02-build_db/reads.seqdb"         input_m4="$rel/09-ovl_filter/ovl.final.m4"         input_m4_phasing_merge="$rel/08-phasing_merge/ovl.phased.m4"         input_reads="/mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/input.fofn"         params_ctg_prefix=ctg         params_num_threads=25         params_config_sh_fn="$rel/01-generate_config/generated.config"         params_log_level="INFO"         params_tmp_dir="/mnt/iscsi01-copy/userdir/Bothrioplana/ipa"             time ipa2-task assemble

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message The params used to generate one or several output files has changed: To inspect which output files have changes, run 'snakemake --list-params-changes'. To trigger a re-run, use 'snakemake -R $(snakemake --list-params-changes)'. Complete log: /mnt/iscsi01-copy/userdir/Bothrioplana/ipa/RUN/.snakemake/log/2022-03-22T115640.011885.snakemake.log ´´´

I've tried running : snakemake --list-params-changes and it returns: Error: no Snakefile found, tried Snakefile, snakefile, workflow/Snakefile, workflow/snakefile

Also: snakemake -R $(snakemake --list-params-changes) returns: Error: you need to specify the maximum number of CPU cores to be used at the same time. If you want to use N cores, say --cores N or -cN. For all cores on your system (be sure that this is appropriate) use --cores all. For no parallelization use --cores 1 or -c1.

So, I've tried specifying number of cores, but it doesn't recognise the parameter. So I am lost here, don't know how to get my run finished. Any clue will be welcome.

Thanks

Marta

MartonaAlvarez commented 1 year ago

Hi,

I've seen that my issue has been closed as resolved, but I didn't get any answer about the last problem I had...

Thanks

Marta

Missatge de Armin Töpfer @.***> del dia dl., 14 de nov. 2022 a les 16:29:

Closed #479 https://github.com/PacificBiosciences/pbbioconda/issues/479 as completed.

— Reply to this email directly, view it on GitHub https://github.com/PacificBiosciences/pbbioconda/issues/479#event-7805388603, or unsubscribe https://github.com/notifications/unsubscribe-auth/AT7BTRABAASXPEVO56X3W3LWIJLFJANCNFSM5QPNKZ6A . You are receiving this because you were mentioned.Message ID: @.*** com>