search

hyle-team / zano

Confidential, Secure, Easy-to-Use
https://zano.org
Other
111 stars 62 forks source link

Zanod crashed on Raspberry PI 4 using lmbd #433

Open apophis1329 opened 10 months ago

apophis1329 commented 10 months ago

Hi,

i compiled zanod from sources following the instructions (using openssl-1.1.1t) - compile worked without any issues ! OS used was Raspberry PI OS: PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=debian

running the freshly compiled bits like this: ./zanod --no-predownload

sync starts and runs for some time before a crash occurs: 2024-Jan-16 00:11:01.135862 [P2P2][95.217.46.49:11121 OUT]>>>>>>>>> sync progress: 200 blocks added, now have 571121 of 2460797 ( 23.21% ) and 1889676 blocks left 2024-Jan-16 00:11:08.904551 [P2P9][95.217.46.49:11121 OUT]>>>>>>>>> sync progress: 199 blocks added, now have 571320 of 2460797 ( 23.22% ) and 1889477 blocks left 2024-Jan-16 00:11:17.055905 [P2P7][95.217.46.49:11121 OUT]>>>>>>>>> sync progress: 200 blocks added, now have 571520 of 2460797 ( 23.22% ) and 1889277 blocks left 2024-Jan-16 00:11:23.271447 [P2P2][95.217.46.49:11121 OUT]Recalculating median fee... 2024-Jan-16 00:11:23.607672 [P2P2][95.217.46.49:11121 OUT]Median fee recalculated for h = 571680 as 0.010000000000 2024-Jan-16 00:11:26.119902 [P2P2][95.217.46.49:11121 OUT]>>>>>>>>> sync progress: 200 blocks added, now have 571720 of 2460799 ( 23.23% ) and 1889079 blocks left 2024-Jan-16 00:11:30.100240 [P2P7][95.217.46.49:11121 OUT][ERROR] Location: [set] @ /COINS/ZANO/zano/src/common/db_backend_lmdb.cpp:336 STACK ./zanod(_ZN5tools13get_callstackB5cxx11Ev+0x14) [0x557c1bdf18] ./zanod(_ZN5tools2db15lmdb_db_backend3setEmPKcmS3_m+0x164) [0x557c45f094] ./zanod(+0x30fe24) [0x557c3bfe24] ./zanod(_ZN8currency18blockchain_storage31update_spent_tx_flags_for_inputERKN6crypto4hashEmb+0x1f8) [0x557c3722b8] ./zanod(_ZN8currency18blockchain_storage31update_spent_tx_flags_for_inputEmmb+0xc8) [0x557c37296c] ./zanod(+0x2d940c) [0x557c38940c] ./zanod(_ZN8currency18blockchain_storage26add_transaction_from_blockERKNS_11transactionERKN6crypto4hashES7_mm+0x24c) [0x557c3898ec] ./zanod(_ZN8currency18blockchain_storage26handle_block_to_main_chainERKNS_5blockERKN6crypto4hashERNS_26block_verification_contextE+0xce0) [0x557c3a2ca0] ./zanod(_ZN8currency18blockchain_storage13add_new_blockERKNS_5blockERNS_26block_verification_contextE+0x6b4) [0x557c3a6b04] ./zanod(_ZN8currency4core13add_new_blockERKNS_5blockERNS_26block_verification_contextE+0x50) [0x557c3db454] ./zanod(_ZN8currency4core21handle_incoming_blockERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERNS_26block_verification_contextEb+0x1cc) [0x557c3dd99c] ./zanod(_ZN8currency27t_currency_protocol_handlerINS_4coreEE27handle_response_get_objectsEiRNS_27NOTIFY_RESPONSE_GET_OBJECTS7requestERNS_27currency_connection_contextE+0x145c) [0x557c2e3dbc] ./zanod(+0x3f8a58) [0x557c4a8a58] ./zanod(+0x3f1684) [0x557c4a1684] ./zanod(+0x1d8cf4) [0x557c288cf4] ./zanod(_ZN4epee9net_utils10connectionINS_5levin22async_protocol_handlerIN8nodetool24p2p_connection_context_tIN8currency27currency_connection_contextEEEEEE11handle_readERKN5boost6system10error_codeEm+ 0xa8) [0x557c278668] ./zanod(_ZN5boost4asio6detail18completion_handlerINS1_17rewrapped_handlerINS1_7binder2INS1_15wrapped_handlerINS0_10io_context6strandENS_3_bi6bind_tIvNS_4_mfi3mf2IvN4epee9net_utils10connectionINSC_5lev in22async_protocol_handlerIN8nodetool24p2p_connection_context_tIN8currency27currency_connection_contextEEEEEEERKNS_6system10error_codeEmEENS8_5list3INS8_5valueINS_10shared_ptrISN_EEEEPFNS_3argILi1EEEv EPFNSY_ILi2EEEvEEEEENS1_26is_continuation_if_runningEEESP_mEES16_EEE11do_completeEPvPNS1_19scheduler_operationESR_m+0x124) [0x557c2abcb4] ./zanod(_ZN5boost4asio6detail23reactive_socket_recv_opINS0_17mutable_buffers_1ENS1_15wrapped_handlerINS0_10io_context6strandENS_3_bi6bind_tIvNS_4_mfi3mf2IvN4epee9net_utils10connectionINSB_5levin22asyn c_protocol_handlerIN8nodetool24p2p_connection_context_tIN8currency27currency_connection_contextEEEEEEERKNS_6system10error_codeEmEENS7_5list3INS7_5valueINS_10shared_ptrISM_EEEEPFNS_3argILi1EEEvEPFNSX_I Li2EEEvEEEEENS1_26is_continuation_if_runningEEENS1_18io_object_executorINS0_8executorEEEE11do_completeEPvPNS1_19scheduler_operationESQ_m+0x618) [0x557c29e4a8] ./zanod(+0x45ec7c) [0x557c50ec7c] ./zanod(_ZN4epee9net_utils18boosted_tcp_serverINS_5levin22async_protocol_handlerIN8nodetool24p2p_connection_context_tIN8currency27currency_connection_contextEEEEEE13worker_threadEv+0x2dc) [0x557c257fa 0]

zanod.log_crashing.gz

cryptozoidberg commented 10 months ago

@apophis1329 thanks for reporting the issue, we'll work on it as soon as we finish Zarcanum transition

Derloda commented 6 months ago

The issue is caused by the outdated liblmdb version used here which do not support 16K page: https://bugs.openldap.org/show_bug.cgi?id=9806

It is already fixed upstream, syncing the repository is most likely the only thing that needs to be done. It would also enable a native apple silicon build, which would greatly improves performance.

Edit: Unfortunately it also probably means the database couldn't be universally shared between 16K/4K page systems. In that case, enforcing --no-predownload (or providing a compatible database) would be mandatory on 16K page systems.

cryptozoidberg commented 6 months ago

@Derloda thanks for investigation of the issue, it's likely indeed time to upstream lmdb library, just need to make sure the new version would work smooth on all platform and won't behave different from old one in terms of blocks/txs processing (to avoid potential network splits). We'll plan this task for one of the nearest sprints.

apophis1329 commented 6 months ago

Please let me know if you have a version ready for testing - very happy to help with testing on my different Raspis :-)

Derloda commented 4 months ago

Any update regarding the issue? It would be great to be able to compile natively for apple silicon.

cryptozoidberg commented 4 months ago

We had few conversations regarding support of Raspberry PI, and honestly i still not sure if it's good idea, mostly because of hardware limitations, especially with Zarcanum's transactions which is more heavier from computational perspective. Of course i'm not controlling this, and anyone can make Zano Raspberry-compatible, i just don't feel good about having a lot of this type of devices as Zano nodes. Regarding apple silicon - is there any problems with this? We have codebase compiled for mobile app in native library for M-processors with no issues (https://github.com/hyle-team/zano_native_lib)

Derloda commented 4 months ago

For Apple Silicon, yes it's the same issue (lmdb in Zanod). It can't even synchronize correctly because of that, wallet usage is not the issue.

sowle commented 3 months ago

I took a look at the issue. Indeed, the issue 9806 was fixed in liblmdb, effective starting with version 0.9.30 by this commit.

However, I can't see how I can manually set page size if I'd like to convert a DB with one page size into another. Key threshold size is always defiled as:

            /* Threshold number of keys considered "small" */
            keythresh = env->me_psize >> 7;

(https://github.com/LMDB/lmdb/blob/LMDB_0.9.31/libraries/liblmdb/mdb.c#L8766)

Where me_psize is derived from the system page size and cannot be changed, if I understand it correctly.

UPD. Note: ver. 0.9.31 is the latest on github mirror, but isn't the latest in openldap git, where 0.9.33 exists: https://git.openldap.org/openldap/openldap/-/tags/LMDB_0.9.33 However, it's still unclear is it possible to create a 16k page DB on 4k page system.