search

HorizenOfficial / zen_archived

TLS integration and more!
https://zensystem.io
Other
128 stars 44 forks source link

zend Seg Fault at block 139200 #84

Open AYANE-69 opened 7 years ago

AYANE-69 commented 7 years ago

Describe the issue

zend predictably seg faults attempting to process block 139200

Can you reliably reproduce the issue?

  1. Install zencash from Arch Linux AUR
  2. Start zend
  3. Wait until block height reaches 139200
  4. Seg fault

Expected behaviour

Initial sync of block chain from nothing to current live state

Actual behaviour + errors

zend syncs to 139200 then seg faults No useful output on command line debug.log contains no errors

The version of Zen you were using:

Zen Daemon version v2.0.10-7a082ad

Machine specs:

Any extra information that might be useful in the debugging process.

debug.log shows no error, final line: 2017-10-27 00:02:25 UpdateTip: new best=00000001ea53c09a45e3f097ba8f48a4c117b5b368031c4eb2fa02cb5a84c99e height=139200 log2_work=45.637116 tx=355208 date=2017-07-21 19:42:01 progress=0.332129 cache=4.4MiB(1637tx)

WaveringAna commented 7 years ago

Eek thats not good, whats special about that block is thats when our first HF activated, can you send your compiled zend to info@zensystem.io for debugging?

WaveringAna commented 7 years ago

can you see if the development branch fixes it?

tarrenj commented 7 years ago

Have there been any updates on this? I've got an Arch system back home, I'll build ZEN off the AUR when I get back to test...

AYANE-69 commented 7 years ago

I had issues building development due to boost library issues. I will take a look at the PKGBUILD on the AUR and see if I can modify it to build from the development branch as I know it does some patching to solve the boost issues.

AYANE-69 commented 7 years ago

OK I edited the PKGBUILD replacing the commit it was building from with 728fb4259cfa663d594ba494b229e8dc86f925b9 which is the last commit is see in the development branch. The package built fine this time.

However, the bug remains. Still seg faults at 139200.

AYANE-69 commented 7 years ago

So I changed PKGBUILD again to build development tree from head as I wasn't sure I was building the right thing previously. Cleared all data and began sync. Still seg faults at 139200.

asbachb commented 6 years ago

@aayanl Is there any information how the official debian package is build (gcc version, lib version)? I guess the arch libs are quite never than the official build.

asbachb commented 6 years ago

gdb run, gdb backtrace

Since starting this node 0 seconds ago:
- You have validated no transactions.

Init message: 

[Press Ctrl+C to exit] [Set 'showmetrics=0' to hide]
Since starting this node 6 seconds ago:
- You have validated 110 transactions!

Since starting this node 7 seconds ago:
- You have validated 538 transactions!

Init message: Loading addresses...

[Press Ctrl+C to exit] [Set 'showmetrics=0' to hide]
[New Thread 0x7fffccef0700 (LWP 2066)]
[New Thread 0x7fffc7fff700 (LWP 2067)]
[New Thread 0x7fffc77fe700 (LWP 2068)]
           Block height | 136116
            Connections | 1 (TLS: 1)
  Network solution rate | 6444298 Sol/s

Since starting this node 18 seconds ago:
- You have validated 745 transactions!

[Press Ctrl+C to exit] [Set 'showmetrics=0' to hide]
[New Thread 0x7ffff7fe87c0 (LWP 2075)]
[New Thread 0x7fffabfff700 (LWP 2076)]
[Thread 0x7ffff7fe87c0 (LWP 2075) exited]
[New Thread 0x7fffab7fe700 (LWP 2077)]
           Block height | 136116
            Connections | 1 (TLS: 1)
           Block height | 139139
            Connections | 7 (TLS: 7)
  Network solution rate | 4904032 Sol/s

Since starting this node 1 minutes, 2 seconds ago:
- You have validated 18353 transactions!

[Press Ctrl+C to exit] [Set 'showmetrics=0' to hide]
[New Thread 0x7fffabfff700 (LWP 2089)]
[New Thread 0x7fffab7fe700 (LWP 2090)]
[New Thread 0x7fffaaffd700 (LWP 2091)]

Thread 2 "zcash-scriptch" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff41df700 (LWP 2053)]
0x00007ffff4bf62c7 in memcpy@GLIBC_2.2.5 () from /usr/lib/libc.so.6
(gdb) backtrace
#0  0x00007ffff4bf62c7 in memcpy@GLIBC_2.2.5 () from /usr/lib/libc.so.6
#1  0x00005555557af39c in ?? ()
#2  0x00005555557b1926 in ?? ()
#3  0x00005555557b5364 in ?? ()
#4  0x00005555555f22a9 in ?? ()
#5  0x000055555563462d in ?? ()
#6  0x00007ffff749a44f in ?? () from /usr/lib/libboost_thread.so.1.65.1
#7  0x00007ffff4f1508a in start_thread () from /usr/lib/libpthread.so.0
#8  0x00007ffff4c4c42f in clone () from /usr/lib/libc.so.6
cronicc commented 6 years ago

This most likely related to arch using libboost 1.65.1, zend depends on 1.62.0. There were some ABI changes and these patches are also not applied.

asbachb commented 6 years ago

Tried to build the aur source with --enable-debug option in order to get a more detailed stacktrace which fails entirely:

  CXXLD    zen-gtest
  CXX      zcash/GenerateParams.o
In file included from /usr/include/bits/libc-header-start.h:33:0,
                 from /usr/include/stdint.h:26,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/7.2.1/include/stdint.h:9,
                 from ./compat/endian.h:12,
                 from ./serialize.h:9,
                 from ./zcash/Proof.hpp:4,
                 from ./zcash/JoinSplit.hpp:5,
                 from zcash/GenerateParams.cpp:1:
/usr/include/features.h:376:4: warning: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Wcpp]
 #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
    ^~~~~~~
  CXXLD    zcash/GenerateParams
libbitcoin_util.a(libbitcoin_util_a-sync.o): In function `boost::thread_specific_ptr<std::vector<std::pair<void*, CLockLocation>, std::allocator<std::pair<void*, CLockLocation> > > >::get() const':
/usr/include/boost/thread/tss.hpp:84: undefined reference to `boost::detail::get_tss_data(void const*)'
libbitcoin_util.a(libbitcoin_util_a-sync.o): In function `boost::thread_specific_ptr<std::vector<std::pair<void*, CLockLocation>, std::allocator<std::pair<void*, CLockLocation> > > >::reset(std::vector<std::pair<void*, CLockLocation>, std::allocator<std::pair<void*, CLockLocation> > >*)':
/usr/include/boost/thread/tss.hpp:105: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
libbitcoin_util.a(libbitcoin_util_a-sync.o): In function `boost::thread_specific_ptr<std::vector<std::pair<void*, CLockLocation>, std::allocator<std::pair<void*, CLockLocation> > > >::~thread_specific_ptr()':
/usr/include/boost/thread/tss.hpp:79: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `SetupEnvironment()':
/home/asbachb/.cache/pacaur/zencash/src/zencash-2.0.10_1/src/util.cpp:869: undefined reference to `boost::filesystem::path::imbue(std::locale const&)'
/home/asbachb/.cache/pacaur/zencash/src/zencash-2.0.10_1/src/util.cpp:870: undefined reference to `boost::filesystem::path::imbue(std::locale const&)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::filesystem::path::operator/=(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
/usr/include/boost/filesystem/path.hpp:339: undefined reference to `boost::filesystem::path::operator/=(boost::filesystem::path const&)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::filesystem::path::has_root_directory() const':
/usr/include/boost/filesystem/path.hpp:520: undefined reference to `boost::filesystem::path::root_directory() const'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::filesystem::operator/(boost::filesystem::path const&, boost::filesystem::path const&)':
/usr/include/boost/filesystem/path.hpp:792: undefined reference to `boost::filesystem::path::operator/=(boost::filesystem::path const&)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::filesystem::exists(boost::filesystem::path const&)':
/usr/include/boost/filesystem/operations.hpp:446: undefined reference to `boost::filesystem::detail::status(boost::filesystem::path const&, boost::system::error_code*)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::filesystem::is_directory(boost::filesystem::path const&)':
/usr/include/boost/filesystem/operations.hpp:451: undefined reference to `boost::filesystem::detail::status(boost::filesystem::path const&, boost::system::error_code*)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::filesystem::create_directories(boost::filesystem::path const&)':
/usr/include/boost/filesystem/operations.hpp:566: undefined reference to `boost::filesystem::detail::create_directories(boost::filesystem::path const&, boost::system::error_code*)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::filesystem::create_directory(boost::filesystem::path const&)':
/usr/include/boost/filesystem/operations.hpp:572: undefined reference to `boost::filesystem::detail::create_directory(boost::filesystem::path const&, boost::system::error_code*)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::filesystem::file_size(boost::filesystem::path const&)':
/usr/include/boost/filesystem/operations.hpp:614: undefined reference to `boost::filesystem::detail::file_size(boost::filesystem::path const&, boost::system::error_code*)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::filesystem::system_complete(boost::filesystem::path const&)':
/usr/include/boost/filesystem/operations.hpp:710: undefined reference to `boost::filesystem::detail::system_complete(boost::filesystem::path const&, boost::system::error_code*)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::filesystem::temp_directory_path()':
/usr/include/boost/filesystem/operations.hpp:716: undefined reference to `boost::filesystem::detail::temp_directory_path(boost::system::error_code*)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::thread_specific_ptr<std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >::~thread_specific_ptr()':
/usr/include/boost/thread/tss.hpp:79: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::thread_specific_ptr<std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >::get() const':
/usr/include/boost/thread/tss.hpp:84: undefined reference to `boost::detail::get_tss_data(void const*)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::thread_specific_ptr<std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >::reset(std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >*)':
/usr/include/boost/thread/tss.hpp:105: undefined reference to `boost::detail::set_tss_data(void const*, boost::shared_ptr<boost::detail::tss_cleanup_function>, void*, bool)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::program_options::detail::basic_config_file_iterator<char>::basic_config_file_iterator(std::istream&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, bool)':
/usr/include/boost/program_options/detail/config_file.hpp:155: undefined reference to `boost::program_options::detail::common_config_file_iterator::common_config_file_iterator(std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, bool)'
/usr/include/boost/program_options/detail/config_file.hpp:158: undefined reference to `boost::program_options::detail::common_config_file_iterator::get()'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `void boost::call_once<void (*&)()>(boost::once_flag&, void (*&)())':
/usr/include/boost/thread/pthread/once_atomic.hpp:123: undefined reference to `boost::thread_detail::enter_once_region(boost::once_flag&)'
/usr/include/boost/thread/pthread/once_atomic.hpp:138: undefined reference to `boost::thread_detail::commit_once_region(boost::once_flag&)'
/usr/include/boost/thread/pthread/once_atomic.hpp:134: undefined reference to `boost::thread_detail::rollback_once_region(boost::once_flag&)'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::eof_iterator<boost::program_options::detail::common_config_file_iterator, boost::program_options::basic_option<char> >::increment()':
/usr/include/boost/program_options/eof_iterator.hpp:79: undefined reference to `boost::program_options::detail::common_config_file_iterator::get()'
libbitcoin_util.a(libbitcoin_util_a-util.o): In function `boost::program_options::detail::basic_config_file_iterator<char>::getline(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)':
/usr/include/boost/program_options/detail/config_file.hpp:171: undefined reference to `boost::program_options::to_internal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
libbitcoin_util.a(libbitcoin_util_a-utiltime.o): In function `boost::this_thread::sleep_for(boost::chrono::duration<long, boost::ratio<1l, 1000000000l> > const&)':
/usr/include/boost/thread/pthread/thread_data.hpp:252: undefined reference to `boost::this_thread::hidden::sleep_for(timespec const&)'
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:2155: zcash/GenerateParams] Error 1
make[2]: Leaving directory '/home/asbachb/.cache/pacaur/zencash/src/zencash-2.0.10_1/src'
make[1]: *** [Makefile:5194: all-recursive] Error 1
make[1]: Leaving directory '/home/asbachb/.cache/pacaur/zencash/src/zencash-2.0.10_1/src'
make: *** [Makefile:624: all-recursive] Error 1
==> ERROR: A failure occurred in build().
    Aborting...
Droygon commented 6 years ago

Hi, I have this problem too. Also somedays (yesterday...for example) I get stuck with segmentation faults on block 110,000 (the fork from zclassic), but today it happens on block 139,200 (the mandatory Zencash update). My info: Zen Daemon version v2.0.10-g5ceedb5c OS name + version: Manjaro Linux 17.1.2 (Arch Linux) CPU: Intel i7-920 RAM: 24GB + Swap Disk size: 80TB Disk Type (HD/SDD): SDD Linux kernel version: Linux 4.14.14-1 x86_64 Compiler version: gcc (GCC) 7.2.1 20171224 Relevant packages installed: boost (1.66.0-1), boost-libs (1.66.0-1), boost-65-compat (1.65.1-1) Last lines in .zen/debug.log:

2018-01-21 13:53:36 UpdateTip: new best=00000005c3589e1ee13631c042465ccecdff9f8b2c698e047be20ba984eb788d height=139197 log2_work=45.637063 tx=355197 date=2017-07-21 19:37:27 progress=0.208225 cache=7.8MiB(2285tx) 2018-01-21 13:53:36 UpdateTip: new best=00000001e360aa0bf7eed566094c742b5dae31285863a3f0af2af8b56de3e2d3 height=139198 log2_work=45.637081 tx=355201 date=2017-07-21 19:39:32 progress=0.208228 cache=7.8MiB(2289tx) 2018-01-21 13:53:37 ERROR: AcceptToMemoryPool: joinsplit requirements not met 2018-01-21 13:53:37 ERROR: AcceptToMemoryPool: joinsplit requirements not met 2018-01-21 13:53:37 UpdateTip: new best=000000028d3682a91cb4022ec65c8221838db1d63ceb73b22f5154c2817524a8 height=139199 log2_work=45.637099 tx=355204 date=2017-07-21 19:40:23 progress=0.208229 cache=7.8MiB(2291tx) 2018-01-21 13:53:37 UpdateTip: new best=00000001ea53c09a45e3f097ba8f48a4c117b5b368031c4eb2fa02cb5a84c99e height=139200 log2_work=45.637116 tx=355208 date=2017-07-21 19:42:01 progress=0.208232 cache=7.8MiB(2295tx) 2018-01-21 13:58:22

Please note that the error: 'ERROR: AcceptToMemoryPool: joinsplit requirements not met' shows up multiple times in the middle of the log file as well, so it doesn't seem to me to be the cause of the fault.

P.S. Should I open a different ticket for the segmentation fault at block 110,000?

daira commented 6 years ago

Please note that the error: 'ERROR: AcceptToMemoryPool: joinsplit requirements not met' shows up multiple times in the middle of the log file as well, so it doesn't seem to me to be the cause of the fault.

It's not, that's a red herring that also shows up for Zcash.