search

PacificBiosciences / pbbioconda

PacBio Secondary Analysis Tools on Bioconda. Contains list of PacBio packages available via conda.
BSD 3-Clause Clear License
251 stars 44 forks source link

falcon_unzip.py failing during polishing (falconc) #198

Closed Neato-Nick closed 4 years ago

Neato-Nick commented 5 years ago

Operating system Which operating system and version are you using?

CentOS Linux release 7.2.1511

Package name Which package / tool is causing the problem? Which version are you using, use tool --version. Have you updated to the latest version conda update package? Have you updated the complete env by running conda update --all? Have you ensured that your channel priorities are set up according to the bioconda recommendations at https://bioconda.github.io/#set-up-channels?

falcon_unzip.py: falcon-unzip 1.3.5 falcon-kit 1.5.2 pypeflow 2.3.0

falconc --version returns nothing.

Conda environment What is the result of conda list? (Try to paste that between triple backticks.)

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
bcftools                  1.9                  ha228f0b_4    bioconda
bedtools                  2.28.0               hdf88d34_0    bioconda
blas                      1.0                         mkl  
blasr                     5.3.3                h707fff8_0    bioconda
blasr_libcpp              5.3.3                h707fff8_0    bioconda
bwa                       0.7.17               h84994c4_5    bioconda
bzip2                     1.0.8                h7b6447c_0  
ca-certificates           2019.9.11            hecc5488_0    conda-forge
certifi                   2019.9.11                py37_0    conda-forge
curl                      7.65.2               hbc83047_0  
decorator                 4.4.0                    py37_1  
falcon-kit                1.5.2                    pypi_0    pypi
falcon-phase              1.2.0                    pypi_0    pypi
falcon-unzip              1.3.5                    pypi_0    pypi
future                    0.17.1                   py37_0  
hdf5                      1.10.4               hb1b8bf9_0  
htslib                    1.9                  ha228f0b_7    bioconda
intel-openmp              2019.4                      243  
krb5                      1.16.1               h173b8e3_7  
libcurl                   7.65.2               h20c2e04_0  
libdeflate                1.0                  h14c3975_1    bioconda
libedit                   3.1.20181209         hc058e9b_0  
libffi                    3.2.1                hd88cf55_4  
libgcc-ng                 9.1.0                hdf63c60_0  
libgfortran-ng            7.3.0                hdf63c60_0  
libssh2                   1.8.2                h1ba5d50_0  
libstdcxx-ng              9.1.0                hdf63c60_0  
minimap2                  2.17                 h84994c4_0    bioconda
mkl                       2019.4                      243  
mkl_fft                   1.0.12           py37ha843d7b_0  
mkl_random                1.0.2            py37hd81dba3_0  
mummer4                   4.0.0beta2      pl526hf484d3e_4    bioconda
ncurses                   6.1                  he6710b0_1  
networkx                  2.3                        py_0  
nim-falcon                0.0.0                         0    bioconda
numpy                     1.16.4           py37h7e9f1db_0  
numpy-base                1.16.4           py37hde5b4d6_0  
openssl                   1.1.1c               h516909a_0    conda-forge
pb-assembly               0.0.6                    py37_5    bioconda
pb-dazzler                0.0.0                h470a237_0    bioconda
pb-falcon                 2.2.0            py37hed50d52_0    bioconda
pb-falconc                0.1.1                hc3dfafe_0    bioconda
pbbam                     0.23.0               h88345a9_0    bioconda
pbcopper                  0.4.2                h3f2a3d2_1    bioconda
pbgcpp                    0.0.1                         0    bioconda
pbmm2                     1.0.0                ha888412_0    bioconda
pcre                      8.41              hf484d3e_1003    conda-forge
perl                      5.26.2               h14c3975_0  
pip                       19.1.1                   py37_0  
pysam                     0.15.2           py37h4b7d16d_3    bioconda
python                    3.7.3                h0371630_0  
python-edlib              1.2.4.post1      py37h6bb024c_0    bioconda
python-intervaltree       3.0.2                      py_0    bioconda
python-msgpack            0.6.1            py37h6bb024c_0    bioconda
python-sortedcontainers   2.1.0                      py_0    bioconda
racon                     1.4.0                he860b03_0    bioconda
readline                  7.0                  h7b6447c_5  
samtools                  1.9                 h8571acd_11    bioconda
setuptools                41.0.1                   py37_0  
sqlite                    3.29.0               h7b6447c_0  
tk                        8.6.8                hbc83047_0  
wheel                     0.33.4                   py37_0  
xz                        5.2.4                h14c3975_4  
zlib                      1.2.11               h7b6447c_3

Describe the bug A clear and concise description of what the bug is.

Unzipping fails during the polishing step. Speifically, at the 4-polish/track-reads/.

Error message Paste the error message / stack.

[30403]finished run_tr_stage1('../../0-rawreads/build/raw_reads.db', /nfs2/hts/pacbio/jobs_root/001/001100/tasks/falcon_ns2.tasks.task_falcon0_dazzler_lamerge_apply_jobs...(126), 2500, 30)
# That writes a msgpack partial for each raw_reads.*.las file.
# Default n_core is n_cpu/2. TODO: Configure --n-core
# (We also use a proc for LA4Falcon, so this is half.)
# It is actually limited by number of files, but in theory we could use whole machine if we have enough blocks.

abs_rawread_to_contigs=$(readlink -f rawread_to_contigs) #TODO: No readlink
++ readlink -f rawread_to_contigs
+ abs_rawread_to_contigs=/dfs/BPP/Grunwald_Lab/home/carleson/genome_assembly/PacBio/ramorum/PR-18-069/4-polish/track-reads/rawread_to_contigs
cwd=$(pwd)
++ pwd
+ cwd=/dfs/BPP/Grunwald_Lab/home/carleson/genome_assembly/PacBio/ramorum/PR-18-069/4-polish/track-reads
cd ../..
+ cd ../..
falconc rr-hctg-track2 --read-to-contig-map=${cwd}/read_to_contig_map --output=${abs_rawread_to_contigs} --partials-fn=${cwd}/partials.json
+ falconc rr-hctg-track2 --read-to-contig-map=/abs/path/to/assembly/4-polish/track-reads/read_to_contig_map --output=/abs/path/to/assembly/4-polish/track-reads/rawread_to_contigs --partials-fn=/abs/path/to/assembly/4-polish/track-reads/partials.json
Unknown subcommand "rr-hctg-track2".  It is not similar to defined subcommands.

I opened an issue in falcon-unzip but figured it might fit better here, now that I've got a work around. I worked around this bug by modifying and re-running template.sh in 4-polish/track-reads to call an older executable instead of the falconc tool.

fc_rr_hctg_track2.exe --read-to-contig-map=${{cwd}}/read_to_contig_map --output=${{abs_rawread_to_contigs}} --partials-fn=${{cwd}}/partials.json

pb-cdunn commented 5 years ago

We have

nim-falcon                0.0.1                h2f8a211_0    bioconda
pb-assembly               0.0.6                    py37_5    bioconda
pb-dazzler                0.0.1                h516909a_0    bioconda
pb-falcon                 2.2.0            py37hed50d52_0    bioconda
# no pb-falconc

The test for nim-falcon-0.0.1 actually verifies that falconc rr-hctg-track2 -h works.

You have

nim-falcon                0.0.0                         0    bioconda
pb-assembly               0.0.6                    py37_5    bioconda
pb-dazzler                0.0.0                h470a237_0    bioconda
pb-falcon                 2.2.0            py37hed50d52_0    bioconda
pb-falconc                0.1.1                hc3dfafe_0    bioconda

Something is fishy. You should not have pb-falconc (with the "c") at all. falconc should come from nim-falcon. You might need to remove pb-falconc from your environment. Then try conda update --all

zeeev commented 4 years ago

Hi @Neato-Nick,

Was this issue resolved? If not, please feel free to re-open the issue.

Neato-Nick commented 4 years ago

I did not test updating the software to solve the issue. I was able to get all my samples through using my workaround before the versioning issue was pointed out to me