search

ucrbioinfo / Chimericognizer

Chimericognizer is a novel and accurate chimeric contigs correction tool based on optical maps.
8 stars 1 forks source link

RefAligner not running #1

Open dcopetti opened 5 years ago

dcopetti commented 5 years ago

Hello, I was able to install Chimericognizer and the dependencies, though RefAligner is giving me a problem. I downloaded the AVX version (as suggested here), and when I run Chimericognizer with the test data, I get this error:

$ python -x /home/copettid/bin/Bionano_tools_AVX/RefAligner -y /home/copettid/bin/fa2cmap_multi.pl -o /home/copettid/Rabiosa_v2/chimericognizer/output_test -i ./examples/contig_file_list.example.txt -m ./examples/optmap_list.example.txt -p 8 -a 1.5 -b 1.0 -d 25 -e 50000 -h 50000 -r 80000
  File "/home/copettid/bin/Bionano_tools_AVX/RefAligner", line 2
SyntaxError: Non-ASCII character '\x97' in file /home/copettid/bin/Bionano_tools_AVX/RefAligner on line 3, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

If I run with my own data, I get this error, pointing again that RefAligner folder is missing:

[...]
Contig[2167177]:              0.001 (MB)          0.000 nick(s)/100KB   40.05%    0.00%
Contig[2167178]:              0.001 (MB)          0.000 nick(s)/100KB   34.18%    0.00%

================================Summary================================
Total contigs processed: 2167178
Total length of the contigs: 9502250028
Global nick frequency:  15.703 nick(s) /100KB
Global GC percentage:   43.832%
Global N percentage:    1.495%
=======================================================================
Appending stdout to /home/copettid/Rabiosa_v2/chimericognizer/output_1/refaligner/input_contigs_BspTI_BNG_VS_seq.stdout
Appending stderr to /home/copettid/Rabiosa_v2/chimericognizer/output_1/refaligner/input_contigs_BspTI_BNG_VS_seq.stdout
---------------read .xmap file-------------------
Traceback (most recent call last):
  File "./scripts/main.py", line 114, in <module>
    main()
  File "./scripts/main.py", line 112, in main
    chimeric_removal(fasta_list_file, output_dir, DIGEST, REFALIGNER,  optmap_list_file, num_threads, threshold_1, threshold_2, threshold_3, threshold_4, threshold_5, threshold_6)
  File "/home/copettid/bin/Chimericognizer/scripts/chimeric_removal.py", line 78, in chimeric_removal
    chimeric_removal_onetime(REFALIGNER, DIGEST, output_dir, optmap_list, optmap_type_list, input_merged_file_chname_long, final_nochi_file, num_threads, threshold_1, threshold_2, threshold_3, threshold_4, threshold_5, threshold_6)
  File "/home/copettid/bin/Chimericognizer/scripts/chimeric_removal_onetime.py", line 32, in chimeric_removal_onetime
    chimeric(out_list, out2_list, chimeric_sub_dir, threshold_1, threshold_2, threshold_3, threshold_4, threshold_5, threshold_6)
  File "/home/copettid/bin/Chimericognizer/scripts/chimeric.py", line 338, in chimeric
    chimeric_pairs, scaling, alms = pre_process(i, myfile, myfile2, output_dir, min_confidence, minrefoverhang, minqryoverhang)
  File "/home/copettid/bin/Chimericognizer/scripts/chimeric.py", line 99, in pre_process
    with open(myfile+'.xmap', 'rb') as csvfile:
IOError: [Errno 2] No such file or directory: '/home/copettid/Rabiosa_v2/chimericognizer/output_1/refaligner/input_contigs_BspTI_BNG_VS_seq.xmap'

Is there any other step I am missing about how to install it? I am running it on a Fedora v28 OS, but I get the same error also on a CentOS v7 with Python 2.7.5. Thanks, Dario

dcopetti commented 5 years ago

Some more thoughts: I am using a single map made with the DLE chemistry, BspTI (CTTAAG) is the enzyme. The data was collected on Saphyr, the assembly run in BionanoAccess. -m is a text file with a single line looking like this: BspTI /path/to/DNassembly/output/contigs/exp_refineFinal1/EXP_REFINEFINAL1.cmap separated by a tab. If I separe the field by a space, but I get the same error. How should that field be separated? I added "BspTI" => "CTTAAG" to fa2cmap_multi.pl, that runs normally.

Looking at the stdout file, I see that the header looks for BspQI:

# CMAP File Version:    0.1
# Label Channels:       1
# Nickase Recognition Site 1:   GCTCTTC
# Enzyme 1:     BspQI
# Number of Consensus Nanomaps: 184229

and it looks a bit different than the header of my .cmap file:

# hostname=saphyr1c
# $ cd /home/bionano; /home/bionano/tools/pipeline/1.0/RefAligner/1.0/RefAligner -if /home/bionano/access/local/jobs/1198184/output/contigs/curContigs -merge -o /home/bionano/access/local/jobs/1198184/output/contigs/exp_refineFinal1/EXP_REFINEFINAL1 -f -stdout -stderr
# CompileDir= /home/users4/tanantharaman/branches/7915/7915.7989rel/7915.7989rel CompileCmd=/opt/gcc-5.4.0/bin/g++ -B/opt/binutils-2.27  -fopenmp -Ofast -mavx2 -mfpmath=sse -DUSE_PFLOAT=1 -DUSE_RFLOAT=1 -DUSE_SSE=1 -DUSE_MFLOAT=1 -DUSE_EPOW=1 -I/opt/glibc-2.24/include -DUSE_STATIC -freciprocal-math -fno-signed-zeros -fno-trapping-math -DRELEASE=1 -std=gnu++98  -Wl,--dynamic-linker=/opt/glibc-2.24/lib/ld-linux-x86-64.so.2 -Wl,--rpath=/opt/glibc-2.24/lib -Wl,--rpath=/opt/gcc-5.4.0/lib64 -lrt -lmvec -L/opt/glibc-2.24/lib -L/opt/gcc-5.4.0/lib64 -static -s SVNversion=7989 $Header: http://svn.bnm.local:81/svn/Informatics/RefAligner/branches/7915/RefAligner.cpp 7959 2018-10-09 04:21:44Z tanantharaman $
# FLAGS: USE_SSE=1 USE_AVX=1 USE_MIC=0 USE_PFLOAT=1 USE_RFLOAT=1 USE_MFLOAT=1 USE_EPOW=1 DEBUG=1 VERB=1
# CMAP File Version:    0.2
# Label Channels:       1
# Nickase Recognition Site 1:   cttaag;green_01
# Number of Consensus Maps:     268

did you try running Chimericognizer on a map made with DLE chemistry and assembled on BionanoAccess?

dcopetti commented 5 years ago

There you go, from the last line of output_1/refaligner/input_contigs_BspTI_BNG_VS_seq.stdout : CMAP File Version 0.2 in /home/copettid/lolium_bionano/BNaccess_outputs/DN_1198184_250_12/output/contigs/exp_refineFinal1/EXP_REFINEFINAL1.cmap not supported (Only 0.1 is supported) but this still does not explain why the test run fails. any clue?

pwhhero commented 5 years ago

Hi Dario, Thanks for your interest. I carefully read the error information you provide, but I haven't figured out how could these errors happen. Refaligner is a C++ executable provided by Bionano company, and I use os.system() to run it in my python script, which means the running of Refaligner should not be related with python. So I have no idea why the error information ask you to look at some python document for details. Seems the system is looking at each line of Refaligner executable, which is also weird to me. If you want to check if the Refaligner is working on you computer, you could run it independently without Chimericognizer.

Weihua

dcopetti commented 5 years ago

Hi, Running RefAligner directly tells me this:

$ ./RefAligner
# hostname=kp141-242
# $ cd /home/copettid/bin/Bionano_tools_AVX; ./RefAligner
# CompileDir= /home/users3/tanantharaman/branches/4794/4794.5122/4794.5122 CompileCmd=/opt/gcc-4.9.2TSAN/bin/g++ -fopenmp -Ofast -fno-associative-math -mavx -mfpmath=sse -DUSE_PFLOAT=1 -DUSE_RFLOAT=1 -DUSE_SSE=1 -I/home/users/tanantharaman/amdlibm-3-0-2/include -DREPLACE_WITH_AMDLIBM -DUSE_STATIC -flto -DRELEASE=1 -lrt -L/home/users/tanantharaman/amdlibm-3-0-2/lib/static -lamdlibm -L/opt/gcc-4.9.2TSAN/lib64 -s -static SVNversion=5122 $Header: http://svn.bnm.local:81/svn/Informatics/RefAligner/branches/4794/RefAligner.cpp 4916 2016-05-10 18:12:38Z kbhakta $
# FLAGS: USE_SSE=1 USE_AVX=1 USE_MIC=0 USE_PFLOAT=1 USE_RFLOAT=1 DEBUG=1 VERB=1
No map input file specified: only -merge supported with null input map set

Can you confirm that Chimericognizer should work also with v2 cmap files? I am in contact with Bionano support as well, let's see what they say. Thanks!

dcopetti commented 4 years ago

hi @pwhhero Do you have any update on this issue? Can it be that the Chimericognizer scripts work only with the first version of the Bionano HS output files? Thanks, Dario

amvarani commented 3 years ago

Hi @dcopetti I had the same problem here. You should upgrade RefAligner to the latest version. Should work