search

bbuchfink / diamond

Accelerated BLAST compatible local sequence aligner.
GNU General Public License v3.0
1.04k stars 182 forks source link

Illegal Instruction persists using DAA output #303

Closed ReneKat closed 4 years ago

ReneKat commented 5 years ago

DIAMOND v0.9.26.127 build hf79dc1e_0 MacOSX Mojave 10.14.6 Same error as lucventurini "Illegal instruction: 4" I used the guide to install from source as was recommended in previous issues:

wget http://github.com/bbuchfink/diamond/archive/v0.9.26.tar.gz
    tar xzf v0.9.26.tar.gz
    cd diamond-0.9.26
    mkdir bin
    cd bin
    cmake ..
    make install

I am a newbie and usually use conda to install packages. So, I put this file into my miniconda3; then reinstalled diamond to my environment. Which it said it was already up to date. Running my code again: diamond blastx -p 8 -d nr_20190923.dmnd -q MH_Default.contigs.fa --long-reads -f 100 --more-sensitive I get the same error. Any direction would be appreciated. Thank you!

bbuchfink commented 5 years ago

Could you try to build from source again, this time using the parameter -D CMAKE_BUILD_TYPE=Debug for cmake? Please check if the same error will occur with this binary.

ReneKat commented 5 years ago

Hello bbuchfink! Thanks so much for your reply! <3

I used cmake -D CMAKE_BUILD_TYPE=Debug

Got the following error:

`(diamond) MYC:bin Owl$ cmake -D CMAKE_BUILD_TYPE=Debug CMake Warning: No source or binary directory provided. Both will be assumed to be the same as the current working directory, but note that this warning will become a fatal error in future CMake releases.

CMake Error: The source directory "/Users/ME/miniconda3/pkgs/diamond-0.9.26/bin" does not appear to contain CMakeLists.txt. Specify --help for usage, or press the help button on the CMake GUI.`

bbuchfink commented 5 years ago

You have to use cmake -D CMAKE_BUILD_TYPE=Debug .. also while being in the diamond-0.9.26/bin directory.

ReneKat commented 5 years ago

cmake -D CMAKE_BUILD_TYPE=Debug ..

-- The C compiler identification is AppleClang 10.0.1.10010046 -- The CXX compiler identification is AppleClang 10.0.1.10010046 -- Check for working ... ... -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE
-- Configuring done -- Generating done -- Build files have been written to: /Users/ME/miniconda3/pkgs/diamond-0.9.26/bin

make install

Scanning dependencies of target diamond ... [ 20%] Building CXX object CMakeFiles/diamond.dir/src/search/search.cpp.o In file included from /Users/Owl/miniconda3/pkgs/diamond-0.9.26/src/search/search.cpp:20: In file included from /Users/Owl/miniconda3/pkgs/diamond-0.9.26/src/search/hit_filter.h:25: In file included from /Users/Owl/miniconda3/pkgs/diamond-0.9.26/src/search/../dp/smith_waterman.h:25: /Users/Owl/miniconda3/pkgs/diamond-0.9.26/src/search/../dp/dp_matrix.h:72:3: warning: instantiation of variable 'DPmatrix::scores' required here, but no definition is available [-Wundefined-var-template] scores_.clear(); ^ /Users/Owl/miniconda3/pkgs/diamond-0.9.26/src/search/../dp/smith_waterman.h:71:20: note: in instantiation of member function 'DP_matrix::DP_matrix' requested here DP_matrix<_score> dp (slen, qlen, band, padding); ^ /Users/Owl/miniconda3/pkgs/diamond-0.9.26/src/search/hit_filter.h:61:3: note: in instantiation of function template specialization 'smith_waterman<unsigned char, hit_filter>' requested here smith_waterman(query, ^ /Users/Owl/miniconda3/pkgs/diamond-0.9.26/src/search/../dp/dp_matrix.h:137:33: note: forward declaration of template entity is here static threadlocal vector scores, hgap_; ^ /Users/Owl/miniconda3/pkgs/diamond-0.9.26/src/search/../dp/dp_matrix.h:72:3: note: add an explicit instantiation declaration to suppress this warning if 'DPmatrix::scores' is explicitly instantiated in another translation unit scores_.clear(); ^ /Users/Owl/miniconda3/pkgs/diamond-0.9.26/src/search/../dp/dp_matrix.h:74:3: warning: instantiation of variable 'DPmatrix::hgap' required here, but no definition is available [-Wundefined-var-template] hgap_.clear(); ^ /Users/Owl/miniconda3/pkgs/diamond-0.9.26/src/search/../dp/dp_matrix.h:137:42: note: forward declaration of template entity is here static threadlocal vector scores, hgap_; ^ /Users/Owl/miniconda3/pkgs/diamond-0.9.26/src/search/../dp/dp_matrix.h:74:3: note: add an explicit instantiation declaration to suppress this warning if 'DPmatrix::hgap' is explicitly instantiated in another translation unit hgap_.clear(); ^ 2 warnings generated. ... ... [100%] Linking CXX executable diamond [100%] Built target diamond Install the project... -- Install configuration: "Debug" -- Installing: /usr/local/bin/diamond

opened new Terminal window

(diamond) MYC:2019.09.25 Owl$ conda update diamond

Collecting package metadata (current_repodata.json): done Solving environment: done

All requested packages already installed.

(diamond) MYC:2019.09.25 Owl$ diamond blastx -p 8 -d /Users/ME/Results/2019.09.23/20190923.dmnd -q /Users/ME/Results/2019.09.23/MH_Default.contigs.fa --more-sensitive --long-reads -f 100

diamond v0.9.26.127 | by Benjamin Buchfink buchfink@gmail.com Licensed under the GNU GPL https://www.gnu.org/licenses/gpl.txt Check http://github.com/bbuchfink/diamond for updates.

CPU threads: 8

Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1) Temporary directory: Opening the database... [0.000832s] ... ... Building seed filter... [0.370842s] Searching alignments... Illegal instruction: 4`

bbuchfink commented 5 years ago

Please try to run diamond with a debugger attached which will help me see the cause of this crash.

Use lddb /path/to/diamond to start the debugger. The diamond binary should be in the diamond-0.9.26/bin directory where you compiled (with build type debug).

Once in lddb, type r args where args is the command line arguments to diamond.

When the crash occurs, type bt to get a backtrace and send the output to me please.

nextgenusfs commented 4 years ago

Seeing the same error from Bioconda packaged diamond for v0.9.25 and v0.9.26 -- also on Mac OS 10.14.6. Any progress?

#create database
diamond makedb --threads 2 --in uniprot_sprot.fasta --db diamond

#run blastx
diamond blastx --threads 2 -q genome.fasta --db diamond -o diamond.matches.tab -e 1e-10 -k 0 --more-sensitive -f 6 sseqid slen sstart send qseqid qlen qstart qend pident length evalue score qcovhsp qframe
diamond v0.9.26.127 | by Benjamin Buchfink <buchfink@gmail.com>
Licensed under the GNU GPL <https://www.gnu.org/licenses/gpl.txt>
Check http://github.com/bbuchfink/diamond for updates.

#CPU threads: 2
Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)
Temporary directory: 
Opening the database...  [0.000147s]
#Target sequences to report alignments for: unlimited
Opening the input file...  [6.4e-05s]
Opening the output file...  [5e-05s]
Loading query sequences...  [0.531401s]
Masking queries...  [3.70398s]
Building query seed set...  [0.000232s]
Algorithm: Double-indexed
Building query histograms...  [4.63456s]
Allocating buffers...  [0.000101s]
Loading reference sequences...  [0.40821s]
Masking reference...  [10.2913s]
Building reference histograms...  [14.0271s]
Allocating buffers...  [3.8e-05s]
Initializing temporary storage...  [0.001158s]
Processing query chunk 0, reference chunk 0, shape 0, index chunk 0.
Building reference index...  [2.83894s]
Building query index...  [0.833725s]
Building seed filter...  [0.155562s]
Searching alignments... Illegal instruction: 4
bbuchfink commented 4 years ago

It's not easy to work on this because I don't have access to a system that exhibits this error. Do you think you could try what I described above? (Compiling from source with -D CMAKE_BUILD_TYPE=Debug and running Diamond using a debugger?)

nextgenusfs commented 4 years ago

I can try -- not quite so easy with the bioconda build (those are using slightly different compilers I think then what is currently on OSX).

nextgenusfs commented 4 years ago

I get a compilation error with clang. Let me see how being build on Conda. 

jon@Jons-MacBook-Pro:~/Downloads/diamond-0.9.27/bin$ cmake -D CMAKE_BUILD_TYPE=Debug ..
-- The C compiler identification is AppleClang 10.0.1.10010046
-- The CXX compiler identification is AppleClang 10.0.1.10010046
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ZLIB: /usr/lib/libz.dylib (found version "1.2.11") 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE  
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/jon/Downloads/diamond-0.9.27/bin
jon@Jons-MacBook-Pro:~/Downloads/diamond-0.9.27/bin$ make install
Scanning dependencies of target arch_generic
[  1%] Building CXX object CMakeFiles/arch_generic.dir/src/dp/swipe/banded_3frame_swipe.cpp.o
In file included from /Users/jon/Downloads/diamond-0.9.27/src/dp/swipe/banded_3frame_swipe.cpp:19:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/algorithm:644:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/memory:666:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/cassert:21:
In file included from /usr/local/include/assert.h:5:
/usr/local/include/except.h:15:32: error: typedef redefinition with different types ('struct Except_Frame_T *' vs 'Except_Frame_T')
typedef struct Except_Frame_T *Except_Frame_T;
                               ^
/usr/local/include/except.h:15:16: note: previous definition is here
typedef struct Except_Frame_T *Except_Frame_T;
               ^
/usr/local/include/except.h:17:18: error: field has incomplete type 'Except_Frame_T'
  Except_Frame_T prev;
                 ^
/usr/local/include/except.h:16:8: note: definition of 'Except_Frame_T' is not complete until the closing '}'
struct Except_Frame_T {
       ^
In file included from /Users/jon/Downloads/diamond-0.9.27/src/dp/swipe/banded_3frame_swipe.cpp:24:
/Users/jon/Downloads/diamond-0.9.27/src/dp/swipe/target_iterator.h:34:3: error: field of type 'const vector<DpTarget>::const_iterator' (aka 'const __wrap_iter<const DpTarget *>') has private constructor
                subject_begin(subject_begin)
                ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iterator:1422:60: note: declared private here
    _LIBCPP_INLINE_VISIBILITY _LIBCPP_CONSTEXPR_IF_NODEBUG __wrap_iter(iterator_type __x) _NOEXCEPT_DEBUG : __i(__x) {}
                                                           ^
3 errors generated.
make[2]: *** [CMakeFiles/arch_generic.dir/src/dp/swipe/banded_3frame_swipe.cpp.o] Error 1
make[1]: *** [CMakeFiles/arch_generic.dir/all] Error 2
make: *** [all] Error 2
bbuchfink commented 4 years ago

Could you try cloning the repo and use the latest commit?

nextgenusfs commented 4 years ago

Okay, got the same compilation error. Looks like bioconda doing this, perhaps boost isn't being properly linked outside of conda?: 

#!/bin/bash

sed -i.bak 's/-march=native/-march=x86-64/' CMakeLists.txt

mkdir build
cd build

cmake .. \
      -DCMAKE_INSTALL_PREFIX=$PREFIX \
      -DCMAKE_PREFIX_PATH=$PREFIX \
      -DBOOST_NO_SYSTEM_PATHS=on \
      -DCMAKE_OSX_DEPLOYMENT_TARGET=""

cmake --build . --config Release --target install
bbuchfink commented 4 years ago

Diamond does not depend on Boost any more. I'll have to look at the compiler error separately. Could you try downloading the source files for the 0.9.26 release and compile this?

nextgenusfs commented 4 years ago

I'm getting the same compilation error on 0.9.26 and also tried 0.9.24 when I do this locally -- so I'm guessing the error is on my end with the apple compilers. Let me see if I can build the bioconda version with conda-bld and add the debug flag. Although not sure I'll be successful as I'm having some problems compiling other bioconda recipes locally as well. Apple must have made some changes again....

nextgenusfs commented 4 years ago

No luck thus far with the builds -- but I can confirm the same illegal instruction problem exists on 0.9.25 and 0.9.26 on bioconda. However, v0.9.24 on bioconda is working as expected (0.9.24-hbde9309_1).

bbuchfink commented 4 years ago

Another thing you could try is compiling with GCC as described here: https://github.com/bbuchfink/diamond_docs/blob/master/4%20Advanced%20topics.MD

nextgenusfs commented 4 years ago

Okay thanks. I'm using diamond in a package for genome annotation and trying to build a conda recipe, while GCC will certainly work locally it won't work on bioconda for OSX.

Or did you mean compile with GCC and see if error persists on OSX?

bbuchfink commented 4 years ago

Yes, that could help if the error persisted, although I'd expect it not to.

I'm trying to get ahold of a Mac that has this problem so I can fix it but it could take some time.

nextgenusfs commented 4 years ago

Hi @bbuchfink just wanted to let you know that I just compiled the tip version from repo successfully on Mac following the default instructions -- and the illegal instruction error is now fixed. So I'm guessing the next release will fix the issue on conda as well. Thanks!

bbuchfink commented 4 years ago

That's interesting as I have not worked on this issue yet and have no idea what caused this change. But I'm glad that it works now and will consider this issue fixed, but please do speak up if you are still getting these Illegal instruction errors on Mac.

dusmanova commented 4 years ago

Hello, It seems I have some similar problems. I am installing diamond on macOS Catalina with conda install -c bioconda diamond=0.9.28. diamond makedb works well, but blastp doesn't. Here is what I get:

#CPU threads: 4
Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)
Temporary directory:
Opening the database...  [0.000107s]
#Target sequences to report alignments for: 25
Opening the input file...  [5.4e-05s]
Opening the output file...  [4.6e-05s]
Loading query sequences...  [0.000147s]
Masking queries...  [0.001328s]
Building query seed set...  [0.000994s]
Algorithm: Query-indexed
Building query histograms...  [0.000119s]
Allocating buffers...  [1.6e-05s]
Loading reference sequences...  [0.044991s]
Building reference histograms...  [0.032049s]
Allocating buffers...  [1.1e-05s]
Initializing temporary storage...  [0.001674s]
Processing query chunk 0, reference chunk 0, shape 0, index chunk 0.
Building reference index...  [0.034609s]
Building query index...  [0.000396s]
Building seed filter...  [0.000561s]
Searching alignments... zsh: illegal hardware instruction  diamond blastp --db db_Mouse.dmnd --query cds_fasta_aa_Human_test.fasta --out

I've checked several previous versions. v0.9.24 works, v0.9.25, v0.926 and v.0.9.28 - don't. This is unfortunate for me. I was especially interested in later then v.0.9.25 because qseq_gapped and sseq_gapped appeared only in it.

bbuchfink commented 4 years ago

Could you try to clone the github repository and compile from source?

aqvd commented 4 years ago

Hi, I had the same problems when running diamond version 0.9.30 instaled from conda on MacOS Mojave 0.14.6. makedb works fine but when moving to blastx it crashes. This is the output from lldb.


(lldb) target create "//anaconda3/bin/diamond"
Current executable set to '//anaconda3/bin/diamond' (x86_64).
(lldb) blastx -d viralproteins -q VIR_Quality_R1.fq -o VIR_Vs_viralproteins.m8
error: 'blastx' is not a valid command.
(lldb) r blastx -d viralproteins -q VIR_Quality_R1.fq -o VIR_Vs_viralproteins.m8 
Process 49573 launched: '//anaconda3/bin/diamond' (x86_64)
diamond v0.9.30.131 (C) Max Planck Society for the Advancement of Science
Documentation, support and updates available at http://www.diamondsearch.org

#CPU threads: 12
Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)
Temporary directory: 
Opening the database...  [0s]
#Target sequences to report alignments for: 25
Reference = viralproteins.dmnd
Sequences = 373201
Letters = 94844101
Block size = 2000000000
Opening the input file...  [0s]
Opening the output file...  [0s]
Loading query sequences...  [0.293s]
Masking queries...  [0.266s]
Building query seed set...  [0.032s]
Algorithm: Double-indexed
Building query histograms...  [0.061s]
Allocating buffers...  [0s]
Loading reference sequences...  [0.239s]
Masking reference...  [0.789s]
Initializing temporary storage...  [0.001s]
Building reference histograms...  [0.242s]
Allocating buffers...  [0s]
Processing query block 0, reference block 0, shape 0, index chunk 0.
Building reference seed array...  [0.237s]
Building query seed array...  [0.061s]
Computing hash join...  [0.028s]
Building seed filter...  [0.001s]
Searching alignments... Process 49573 stopped
* thread #2, stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
    frame #0: 0x00000001000ddfc4 diamond`Search::ARCH_SSE4_1::search_query_offset(unsigned long, packed_uint40_t const*, std::__1::__wrap_iter<Stage1_hit const*>, std::__1::__wrap_iter<Stage1_hit const*>, Statistics&, Async_buffer<hit>::Iterator&, unsigned int) + 772
diamond`Search::ARCH_SSE4_1::search_query_offset:
->  0x1000ddfc4 <+772>: ud2    
    0x1000ddfc6 <+774>: callq  0x1000e4092               ; symbol stub for: __stack_chk_fail
    0x1000ddfcb <+779>: nopl   (%rax,%rax)

diamond`Search::ARCH_SSE4_1::hit_filter::push:
    0x1000ddfd0 <+0>:   pushq  %rbp
  thread #3, stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
    frame #0: 0x00000001000ddfc4 diamond`Search::ARCH_SSE4_1::search_query_offset(unsigned long, packed_uint40_t const*, std::__1::__wrap_iter<Stage1_hit const*>, std::__1::__wrap_iter<Stage1_hit const*>, Statistics&, Async_buffer<hit>::Iterator&, unsigned int) + 772
diamond`Search::ARCH_SSE4_1::search_query_offset:
->  0x1000ddfc4 <+772>: ud2    
    0x1000ddfc6 <+774>: callq  0x1000e4092               ; symbol stub for: __stack_chk_fail
    0x1000ddfcb <+779>: nopl   (%rax,%rax)

diamond`Search::ARCH_SSE4_1::hit_filter::push:
    0x1000ddfd0 <+0>:   pushq  %rbp
  thread #5, stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
    frame #0: 0x00000001000ddfc4 diamond`Search::ARCH_SSE4_1::search_query_offset(unsigned long, packed_uint40_t const*, std::__1::__wrap_iter<Stage1_hit const*>, std::__1::__wrap_iter<Stage1_hit const*>, Statistics&, Async_buffer<hit>::Iterator&, unsigned int) + 772
diamond`Search::ARCH_SSE4_1::search_query_offset:
->  0x1000ddfc4 <+772>: ud2    
    0x1000ddfc6 <+774>: callq  0x1000e4092               ; symbol stub for: __stack_chk_fail
    0x1000ddfcb <+779>: nopl   (%rax,%rax)

diamond`Search::ARCH_SSE4_1::hit_filter::push:
    0x1000ddfd0 <+0>:   pushq  %rbp
  thread #6, stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
    frame #0: 0x00000001000ddfc4 diamond`Search::ARCH_SSE4_1::search_query_offset(unsigned long, packed_uint40_t const*, std::__1::__wrap_iter<Stage1_hit const*>, std::__1::__wrap_iter<Stage1_hit const*>, Statistics&, Async_buffer<hit>::Iterator&, unsigned int) + 772
diamond`Search::ARCH_SSE4_1::search_query_offset:
->  0x1000ddfc4 <+772>: ud2    
    0x1000ddfc6 <+774>: callq  0x1000e4092               ; symbol stub for: __stack_chk_fail
    0x1000ddfcb <+779>: nopl   (%rax,%rax)

diamond`Search::ARCH_SSE4_1::hit_filter::push:
    0x1000ddfd0 <+0>:   pushq  %rbp
Target 0: (diamond) stopped.
(lldb) bt
* thread #2, stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
  * frame #0: 0x00000001000ddfc4 diamond`Search::ARCH_SSE4_1::search_query_offset(unsigned long, packed_uint40_t const*, std::__1::__wrap_iter<Stage1_hit const*>, std::__1::__wrap_iter<Stage1_hit const*>, Statistics&, Async_buffer<hit>::Iterator&, unsigned int) + 772
    frame #1: 0x00000001000de3a6 diamond`Search::ARCH_SSE4_1::stage2(packed_uint40_t const*, packed_uint40_t const*, std::__1::vector<Stage1_hit, std::__1::allocator<Stage1_hit> > const&, Statistics&, Async_buffer<hit>::Iterator&, unsigned int) + 166
    frame #2: 0x00000001000ddaf0 diamond`Search::ARCH_SSE4_1::stage1(packed_uint40_t const*, unsigned long, packed_uint40_t const*, unsigned long, Statistics&, Async_buffer<hit>::Iterator&, unsigned int) + 832
    frame #3: 0x000000010005e9ac diamond`std::__1::__function::__func<void (*)(packed_uint40_t const*, unsigned long, packed_uint40_t const*, unsigned long, Statistics&, Async_buffer<hit>::Iterator&, unsigned int), std::__1::allocator<void (*)(packed_uint40_t const*, unsigned long, packed_uint40_t const*, unsigned long, Statistics&, Async_buffer<hit>::Iterator&, unsigned int)>, void (packed_uint40_t const*, unsigned long, packed_uint40_t const*, unsigned long, Statistics&, Async_buffer<hit>::Iterator&, unsigned int)>::operator()(packed_uint40_t const*&&, unsigned long&&, packed_uint40_t const*&&, unsigned long&&, Statistics&, Async_buffer<hit>::Iterator&, unsigned int&&) + 44
    frame #4: 0x0000000100090692 diamond`search_worker(std::__1::atomic<unsigned int>*, SeedPartitionRange const*, unsigned int, unsigned long, DoubleArray<packed_uint40_t>*, DoubleArray<packed_uint40_t>*) + 498
    frame #5: 0x00000001000930de diamond`void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (*)(std::__1::atomic<unsigned int>*, SeedPartitionRange const*, unsigned int, unsigned long, DoubleArray<packed_uint40_t>*, DoubleArray<packed_uint40_t>*), std::__1::atomic<unsigned int>*, SeedPartitionRange const*, unsigned int, unsigned long, DoubleArray<packed_uint40_t>*, DoubleArray<packed_uint40_t>*> >(void*) + 62
    frame #6: 0x00007fff5b7ba2eb libsystem_pthread.dylib`_pthread_body + 126
    frame #7: 0x00007fff5b7bd249 libsystem_pthread.dylib`_pthread_start + 66
    frame #8: 0x00007fff5b7b940d libsystem_pthread.dylib`thread_start + 13
}```

I hope this was helpful
bbuchfink commented 4 years ago

Thanks for posting the traceback. Could you please clone the latest version of the repository, compile from source and tell me if the problem persists?

aqvd commented 4 years ago

I tried that also, following the instructions in the installation page. However, compilation fails (this is the output generated during compilation).

$ wget http://github.com/bbuchfink/diamond/archive/v0.9.31.tar.gz
$ tar xzf v0.9.31.tar.gz
$ cd diamond-0.9.31
$ mkdir bin
$ cd bin
$ cmake -D CMAKE_BUILD_TYPE=Debug ..
-- The C compiler identification is AppleClang 10.0.1.10010046
-- The CXX compiler identification is AppleClang 10.0.1.10010046
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ZLIB: /usr/lib/libz.dylib (found version "1.2.11") 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/bin
$ make install
Scanning dependencies of target arch_generic
[  0%] Building CXX object CMakeFiles/arch_generic.dir/src/dp/swipe/banded_3frame_swipe.cpp.o
[  1%] Building CXX object CMakeFiles/arch_generic.dir/src/dp/swipe/swipe.cpp.o
[  2%] Building CXX object CMakeFiles/arch_generic.dir/src/dp/swipe/banded_swipe.cpp.o
[  3%] Building CXX object CMakeFiles/arch_generic.dir/src/search/collision.cpp.o
[  4%] Building CXX object CMakeFiles/arch_generic.dir/src/search/stage1.cpp.o
In file included from /Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/stage1.cpp:21:
In file included from /Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/hit_filter.h:25:
In file included from /Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/smith_waterman.h:25:
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/dp_matrix.h:72:3: warning: 
      instantiation of variable 'Search::ARCH_GENERIC::DP_matrix<unsigned
      char>::scores_' required here, but no definition is available
      [-Wundefined-var-template]
                scores_.clear();
                ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/smith_waterman.h:73:20: note: 
      in instantiation of member function
      'Search::ARCH_GENERIC::DP_matrix<unsigned char>::DP_matrix' requested here
        DP_matrix<_score> dp (slen, qlen, band, padding);
                          ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/hit_filter.h:61:3: note: 
      in instantiation of function template specialization
      'Search::ARCH_GENERIC::smith_waterman<unsigned char,
      Search::ARCH_GENERIC::hit_filter>' requested here
                smith_waterman(query,
                ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/dp_matrix.h:137:38: note: 
      forward declaration of template entity is here
        static thread_local std::vector<sv> scores_, hgap_;
                                            ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/dp_matrix.h:72:3: note: 
      add an explicit instantiation declaration to suppress this warning if
      'Search::ARCH_GENERIC::DP_matrix<unsigned char>::scores_' is explicitly
      instantiated in another translation unit
                scores_.clear();
                ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/dp_matrix.h:74:3: warning: 
      instantiation of variable 'Search::ARCH_GENERIC::DP_matrix<unsigned
      char>::hgap_' required here, but no definition is available
      [-Wundefined-var-template]
                hgap_.clear();
                ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/dp_matrix.h:137:47: note: 
      forward declaration of template entity is here
        static thread_local std::vector<sv> scores_, hgap_;
                                                     ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/dp_matrix.h:74:3: note: 
      add an explicit instantiation declaration to suppress this warning if
      'Search::ARCH_GENERIC::DP_matrix<unsigned char>::hgap_' is explicitly
      instantiated in another translation unit
                hgap_.clear();
                ^
2 warnings generated.
[  5%] Building CXX object CMakeFiles/arch_generic.dir/src/search/stage2.cpp.o
[  6%] Building CXX object CMakeFiles/arch_generic.dir/src/tools/benchmark.cpp.o
[  7%] Building CXX object CMakeFiles/arch_generic.dir/src/dp/swipe/swipe_wrapper.cpp.o
[  8%] Building CXX object CMakeFiles/arch_generic.dir/src/util/tantan.cpp.o
[  8%] Built target arch_generic
Scanning dependencies of target arch_sse4_1
[  9%] Building CXX object CMakeFiles/arch_sse4_1.dir/src/dp/swipe/banded_3frame_swipe.cpp.o
[ 10%] Building CXX object CMakeFiles/arch_sse4_1.dir/src/dp/swipe/swipe.cpp.o
[ 11%] Building CXX object CMakeFiles/arch_sse4_1.dir/src/dp/swipe/banded_swipe.cpp.o
[ 12%] Building CXX object CMakeFiles/arch_sse4_1.dir/src/search/collision.cpp.o
[ 12%] Building CXX object CMakeFiles/arch_sse4_1.dir/src/search/stage1.cpp.o
In file included from /Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/stage1.cpp:21:
In file included from /Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/hit_filter.h:25:
In file included from /Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/smith_waterman.h:25:
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/dp_matrix.h:72:3: warning: 
      instantiation of variable 'Search::ARCH_SSE4_1::DP_matrix<unsigned
      char>::scores_' required here, but no definition is available
      [-Wundefined-var-template]
                scores_.clear();
                ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/smith_waterman.h:73:20: note: 
      in instantiation of member function
      'Search::ARCH_SSE4_1::DP_matrix<unsigned char>::DP_matrix' requested here
        DP_matrix<_score> dp (slen, qlen, band, padding);
                          ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/hit_filter.h:61:3: note: 
      in instantiation of function template specialization
      'Search::ARCH_SSE4_1::smith_waterman<unsigned char,
      Search::ARCH_SSE4_1::hit_filter>' requested here
                smith_waterman(query,
                ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/dp_matrix.h:137:38: note: 
      forward declaration of template entity is here
        static thread_local std::vector<sv> scores_, hgap_;
                                            ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/dp_matrix.h:72:3: note: 
      add an explicit instantiation declaration to suppress this warning if
      'Search::ARCH_SSE4_1::DP_matrix<unsigned char>::scores_' is explicitly
      instantiated in another translation unit
                scores_.clear();
                ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/dp_matrix.h:74:3: warning: 
      instantiation of variable 'Search::ARCH_SSE4_1::DP_matrix<unsigned
      char>::hgap_' required here, but no definition is available
      [-Wundefined-var-template]
                hgap_.clear();
                ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/dp_matrix.h:137:47: note: 
      forward declaration of template entity is here
        static thread_local std::vector<sv> scores_, hgap_;
                                                     ^
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/search/../dp/dp_matrix.h:74:3: note: 
      add an explicit instantiation declaration to suppress this warning if
      'Search::ARCH_SSE4_1::DP_matrix<unsigned char>::hgap_' is explicitly
      instantiated in another translation unit
                hgap_.clear();
                ^
2 warnings generated.
[ 13%] Building CXX object CMakeFiles/arch_sse4_1.dir/src/search/stage2.cpp.o
[ 14%] Building CXX object CMakeFiles/arch_sse4_1.dir/src/tools/benchmark.cpp.o
[ 15%] Building CXX object CMakeFiles/arch_sse4_1.dir/src/dp/swipe/swipe_wrapper.cpp.o
[ 16%] Building CXX object CMakeFiles/arch_sse4_1.dir/src/util/tantan.cpp.o
[ 16%] Built target arch_sse4_1
Scanning dependencies of target diamond
[ 17%] Building CXX object CMakeFiles/diamond.dir/src/run/main.cpp.o
[ 18%] Building CXX object CMakeFiles/diamond.dir/src/basic/config.cpp.o
[ 19%] Building CXX object CMakeFiles/diamond.dir/src/basic/score_matrix.cpp.o
[ 20%] Building CXX object CMakeFiles/diamond.dir/src/data/queries.cpp.o
[ 21%] Building CXX object CMakeFiles/diamond.dir/src/data/reference.cpp.o
[ 22%] Building CXX object CMakeFiles/diamond.dir/src/data/seed_histogram.cpp.o
[ 23%] Building CXX object CMakeFiles/diamond.dir/src/output/daa_record.cpp.o
[ 24%] Building CXX object CMakeFiles/diamond.dir/src/util/command_line_parser.cpp.o
[ 25%] Building CXX object CMakeFiles/diamond.dir/src/util/seq_file_format.cpp.o
[ 25%] Building CXX object CMakeFiles/diamond.dir/src/util/util.cpp.o
[ 26%] Building CXX object CMakeFiles/diamond.dir/src/basic/basic.cpp.o
In file included from /Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/basic/basic.cpp:19:
/Users/aqo/Desktop/ALVARO/Software/diamond-0.9.31/src/basic/value.h:32:20: error: 
      implicit instantiation of undefined template 'std::__1::basic_string<char,
      std::__1::char_traits<char>, std::__1::allocator<char> >'
        const std::string msg;
                          ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:193:32: note: 
      template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_string;
                               ^
1 error generated.
make[2]: *** [CMakeFiles/diamond.dir/src/basic/basic.cpp.o] Error 1
make[1]: *** [CMakeFiles/diamond.dir/all] Error 2
make: *** [all] Error 2

I am quite naive and I have no idea what it's failing, so I used with the conda compiled version. But If you figure any workaround I can try and see if the problem persists with the latest version

bbuchfink commented 4 years ago

Please clone the repository like this:

git clone https://github.com/bbuchfink/diamond.git
cd diamond
mkdir bin
cd bin
cmake ..
make
aqvd commented 4 years ago

Thanks! It seems that everything worked. The compilation and also a test using blastx command. Should be a problem with conda binaries only

bbuchfink commented 4 years ago

Glad that it works now. I did find a problem in the code, which is now fixed in 232967c0e127035eaab0c679b79b15d0a0dccce9. So I will consider this issue fixed now.

snayfach commented 4 years ago

I'm running into this same issue for OSX using diamond version 0.9.32. I get the error Illegal instruction: 4. diamond was installed via conda.

I also followed the instructions above for manual installation (git clone ... make) but got this error:

Scanning dependencies of target arch_generic
[  0%] Building CXX object CMakeFiles/arch_generic.dir/src/dp/swipe/banded_3frame_swipe.cpp.o
In file included from /Users/snayfachbattilana/Desktop/diamond/src/dp/swipe/banded_3frame_swipe.cpp:24:
In file included from /Users/snayfachbattilana/Desktop/diamond/src/dp/swipe/../dp.h:26:
In file included from /Users/snayfachbattilana/Desktop/diamond/src/dp/swipe/../../basic/match.h:26:
In file included from /Users/snayfachbattilana/Desktop/diamond/src/dp/swipe/../../basic/../util/async_buffer.h:26:
In file included from /Users/snayfachbattilana/Desktop/diamond/src/dp/swipe/../../util/io/temp_file.h:23:
In file included from /Users/snayfachbattilana/Desktop/diamond/src/dp/swipe/../../util/io/output_file.h:26:
In file included from /Users/snayfachbattilana/Desktop/diamond/src/dp/swipe/../../util/io/serializer.h:28:
/Users/snayfachbattilana/Desktop/diamond/src/dp/swipe/../../util/io/../system/endianness.h:25:10: fatal error: 'endian.h' file not found
#include <endian.h>
         ^~~~~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/arch_generic.dir/src/dp/swipe/banded_3frame_swipe.cpp.o] Error 1
make[1]: *** [CMakeFiles/arch_generic.dir/all] Error 2
make: *** [all] Error 2
bbuchfink commented 4 years ago

Could you try again compiling from source using the latest github commit? Could you also tell me which macos version you are using?

bbuchfink commented 4 years ago

Another thing you could try is using the option --fast-stage2.

bbuchfink commented 4 years ago

After chasing down a myriad of Apple-specific issues, the latest release hopefully finally works on macOS, at least it does according to the travis ci.