"internal error: entered unreachable code" when starting service on testnet

MikaelBTC commented 1 year ago

Hi, I have encountered the following error when I started the ordhook service on testnet, restarted service still get the same error.

Can someone help me to figure out where is the problem? thanks.

Oct 12 09:25:20.488 INFO Starting service... Oct 12 09:25:20.496 INFO Indexing inscriptions from block #2436517 to block #2532329 Oct 12 09:25:23.604 INFO Processing 100 blocks Oct 12 09:25:23.604 INFO Number of inscriptions in block #2436517 to process: 15 (L1 cache hits: 0, queue: [2436518, 2436519, 2436520, 2436521, 2436522, 2436523, 2436524, 2436525, 2436526, 2436527, 2436528, 2436529, 2436530, 2436531, 2436532, 2436533, 2436534, 2436535, 2436536, 2436537, 2436538, 2436539, 2436540, 2436541, 2436542, 2436543, 2436544, 2436545, 2436546, 2436547, 2436548, 2436549, 2436550, 2436551, 2436552, 2436553, 2436554, 2436555, 2436556, 2436557, 2436558, 2436559, 2436560, 2436561, 2436562, 2436563, 2436564, 2436565, 2436566, 2436567, 2436568, 2436569, 2436570, 2436571, 2436572, 2436573, 2436574, 2436575, 2436576, 2436577, 2436578, 2436579, 2436580, 2436581, 2436582, 2436583, 2436584, 2436585, 2436586, 2436587, 2436588, 2436589, 2436590, 2436591, 2436592, 2436593, 2436594, 2436595, 2436596, 2436597, 2436598, 2436599, 2436600, 2436601, 2436602, 2436603, 2436604, 2436605, 2436606, 2436607, 2436608, 2436609, 2436610, 2436611, 2436612, 2436613, 2436614, 2436615, 2436616], L1 cache len: 0, L2 cache len: 0) Oct 12 09:25:23.980 INFO Completed ordinal number retrieval for Satpoint 0x5303827f29ff1486a05a3012531d2ece294995e8ee834b140620f4d92f2f6794:0:0 (block: #332855:1834831058, transfers: 3250, progress: 1/15, priority queue: true, thread: 10) Oct 12 09:25:23.980 INFO Number of inscriptions in block #2436517 to pre-process: 0 Oct 12 09:25:23.980 INFO Completed ordinal number retrieval for Satpoint 0x8b6d1321fa24e048f510afd34bc27daf9d3fde7ba3a74e0be10a0706b7565110:0:0 (block: #332855:1834816714, transfers: 3246, progress: 2/15, priority queue: true, thread: 12) Oct 12 09:25:23.980 INFO Number of inscriptions in block #2436517 to pre-process: 17 Oct 12 09:25:23.981 INFO Completed ordinal number retrieval for Satpoint 0x8e9b08cf7158c6483701a88c14f2791a6d8fcf474092bdc50ff24de75ee65bdf:0:0 (block: #332855:1834823886, transfers: 3248, progress: 3/15, priority queue: true, thread: 0) Oct 12 09:25:23.985 INFO Completed ordinal number retrieval for Satpoint 0x60a0570889b360de7d77a70745fa67378ec6f150d13999a5890bafbd76d3b05c:0:0 (block: #332855:1834820300, transfers: 3247, progress: 4/15, priority queue: true, thread: 13) Oct 12 09:25:23.985 INFO Completed ordinal number retrieval for Satpoint 0x1b90f5bfc6c135b4cafaa709fd10b246f9704753f7657903d9face1ab8875923:0:0 (block: #332855:1834827472, transfers: 3249, progress: 5/15, priority queue: true, thread: 4) Oct 12 09:25:24.003 INFO Completed ordinal number retrieval for Satpoint 0xbc413c5dd2c45625dc9bd955bff8b982546dce3a2afc04be47abd1b885188ed6:0:0 (block: #386926:481724476, transfers: 5705, progress: 6/15, priority queue: true, thread: 11) Oct 12 09:25:24.127 INFO Completed ordinal number retrieval for Satpoint 0x79f238e97a29fff293fb5c2237c646535017f5663f789f4b8c1bd1e72336e88b:0:0 (block: #356950:1747587355, transfers: 8442, progress: 7/15, priority queue: true, thread: 5) Oct 12 09:25:24.128 INFO Completed ordinal number retrieval for Satpoint 0xf41354158c50a23d5b3f441b560abbf5ff805ea09e0828c21f8425f239791055:0:0 (block: #356950:1747584743, transfers: 8442, progress: 8/15, priority queue: true, thread: 8) Oct 12 09:25:24.128 INFO Completed ordinal number retrieval for Satpoint 0x59fb59ca78961820b7509606116f0ae658bb6a3200b3e7f306a4ef4b22e8ab86:0:0 (block: #356950:1747579519, transfers: 8442, progress: 9/15, priority queue: true, thread: 14) Oct 12 09:25:24.128 INFO Completed ordinal number retrieval for Satpoint 0xdb8d8abc5fe09a367407b4ae2cd365ec9cd9df9899d197496a05f98908633255:0:0 (block: #356950:1747580825, transfers: 8442, progress: 10/15, priority queue: true, thread: 6) Oct 12 09:25:24.128 INFO Completed ordinal number retrieval for Satpoint 0xdb251795985cf44687e8e50219840d56e6f8c0d53b654f60f63620777cad3acc:0:0 (block: #356950:1747586049, transfers: 8442, progress: 11/15, priority queue: true, thread: 3) Oct 12 09:25:24.129 INFO Completed ordinal number retrieval for Satpoint 0xe44e5f4cb7cb0fa9e6c0b698e99c9e1c4c50b659223751db716a9af855bb3786:0:0 (block: #356950:1747582131, transfers: 8442, progress: 12/15, priority queue: true, thread: 2) Oct 12 09:25:24.129 INFO Completed ordinal number retrieval for Satpoint 0x815e62ba8c8ab0c46674a9345acd93da08c72ab37cc7a6537a391854d81ace1d:0:0 (block: #356950:1747578213, transfers: 8441, progress: 13/15, priority queue: true, thread: 1) Oct 12 09:25:24.129 INFO Completed ordinal number retrieval for Satpoint 0x4bbf77556d66e9a83cfe1eb853afa630402adfba58b93c51775a49a7783ab8a2:0:0 (block: #356950:1747583437, transfers: 8442, progress: 14/15, priority queue: true, thread: 7) thread 'Worker' panicked at 'internal error: entered unreachable code', components/ordhook-core/src/core/protocol/satoshi_numbering.rs:237:25 stack backtrace: 0: 0x55cf4e3c7810 - std::backtrace_rs::backtrace::libunwind::trace::h1e9c869ddabe891c at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5 1: 0x55cf4e3c7810 - std::backtrace_rs::backtrace::trace_unsynchronized::h1d559bd09dbc24c9 at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5 2: 0x55cf4e3c7810 - std::sys_common::backtrace::_print_fmt::h3ae709387ceadd9f at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/sys_common/backtrace.rs:65:5 3: 0x55cf4e3c7810 - ::fmt::h19496e227970904a at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/sys_common/backtrace.rs:44:22 4: 0x55cf4e3f521c - core::fmt::rt::Argument::fmt::h18fc21a41543ef0e at /builddir/build/BUILD/rustc-1.72.0-src/library/core/src/fmt/rt.rs:138:9 5: 0x55cf4e3f521c - core::fmt::write::h78fbcccbc63e799c at /builddir/build/BUILD/rustc-1.72.0-src/library/core/src/fmt/mod.rs:1094:21 6: 0x55cf4e3c3c07 - std::io::Write::write_fmt::h1e345f3cfc2298b9 at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/io/mod.rs:1714:15 7: 0x55cf4e3c75c1 - std::sys_common::backtrace::_print::h9d70a06057e2bba5 at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/sys_common/backtrace.rs:47:5 8: 0x55cf4e3c75c1 - std::sys_common::backtrace::print::h7095f0a3537d7cab at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/sys_common/backtrace.rs:34:9 9: 0x55cf4e3c8ede - std::panicking::default_hook::{{closure}}::hfaea52c1f7cf1d21 at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/panicking.rs:269:22 10: 0x55cf4e3c8bee - std::panicking::default_hook::haaa8bc63372a2b35 at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/panicking.rs:288:9 11: 0x55cf4e3c94b4 - std::panicking::rust_panic_with_hook::hc69bf3eb9cc3946e at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/panicking.rs:705:13 12: 0x55cf4e3c9369 - std::panicking::begin_panic_handler::{{closure}}::h656585ee80d93332 at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/panicking.rs:595:13 13: 0x55cf4e3c7c66 - std::sys_common::backtrace::__rust_end_short_backtrace::h01707de9d3720233 at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/sys_common/backtrace.rs:151:18 14: 0x55cf4e3c90c7 - rust_begin_unwind at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/panicking.rs:593:5 15: 0x55cf4d3fb443 - core::panicking::panic_fmt::h968ed060d7c6cd97 at /builddir/build/BUILD/rustc-1.72.0-src/library/core/src/panicking.rs:67:14 16: 0x55cf4d3fb4d3 - core::panicking::panic::h7cbea5846c0c91a6 at /builddir/build/BUILD/rustc-1.72.0-src/library/core/src/panicking.rs:117:5 17: 0x55cf4d6d6ec7 - ordhook::core::protocol::satoshi_numbering::compute_satoshi_number::h90e5a78de084b450 18: 0x55cf4d56513a - std::sys_common::backtrace::__rust_begin_short_backtrace::h5d886db980cc4e2f 19: 0x55cf4d56caae - core::ops::function::FnOnce::call_once{{vtable.shim}}::he7465ca6664149c0 20: 0x55cf4e3cbed2 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce>::call_once::h01c2b804072cdb0f at /builddir/build/BUILD/rustc-1.72.0-src/library/alloc/src/boxed.rs:1993:9 21: 0x55cf4e3cbed2 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce>::call_once::hebf6a458c08a34f6 at /builddir/build/BUILD/rustc-1.72.0-src/library/alloc/src/boxed.rs:1993:9 22: 0x55cf4e3cbed2 - std::sys::unix::thread::Thread::new::thread_start::h8344d8c31e93c732 at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/sys/unix/thread.rs:108:17 23: 0x7f90b287fea5 - start_thread 24: 0x7f90b20a2b0d - clone 25: 0x0 -

MikaelBTC commented 1 year ago

@lgalabru @rafaelcr @diwakergupta @zone117x Can you help me to check this bug? Thanks.

diwakergupta commented 1 year ago

Hi @MikaelBTC , couple of things:

the logs indicate a crash, there's a convenient stack trace and message: thread 'Worker' panicked at 'internal error: entered unreachable code', components/ordhook-core/src/core/protocol/satoshi_numbering.rs:237:25. Likely a bug, I suggest you start there to debug.
we believe in open-source and building in public. HOWEVER, we have a small team and thus our ability to provide technical support for debugging issues for self-hosted / on-premise deployments is very, very limited. I appreciate your patience.
could you share a bit more about your use-case? Have you checked out the Hiro Platform to see if it might fit your needs?

MikaelBTC commented 1 year ago

@diwakergupta Thanks for your relay. I did not do anything special(no special use-case), just used the latest code to compile and started the ordhook service with Bitcoin Testnet, then I encountered this issue. Also I have tried develop branch and v1.0.1, same error too. I think if you try it with Bitcoin Testnet, it will be very easy to reproduce it.

I will be very appreciate it if you can help me to fix it.

Thanks for your great effort for this open source project, it is really amazing.

ordinariusprof commented 1 year ago

we are having the same issue. any progress or hints on this?

lgalabru commented 1 year ago

@MikaelBTC / @ordinariusprof which version of bitcoind are you guys running?

ordinariusprof commented 1 year ago

v24.0.1

MikaelBTC commented 1 year ago

@MikaelBTC / @ordinariusprof which version of bitcoind are you guys running?

Mine is 25.0, thanks @lgalabru

ordinariusprof commented 1 year ago

I can see these logs that don't make much sense because my testnet bitcoin instance has these blocks, I've tried to get these blocks with rpc and it works but I think ordhook is not able to save them into one of the DBs.

Nov 06 22:33:03.266 INFO Block #54504 saved to disk Nov 06 22:33:31.820 WARN unable to fetch block #00000000005991f9f4a17b344907af42ae5e785b6c7e0943a50cdb2975eeb746: will retry in a few seconds (attempt #2). Nov 06 22:33:33.325 WARN unable to fetch block #0000000001725caeac94b12c99a303a83fa987980700cc7a3b6dd472eb5f0d82: will retry in a few seconds (attempt #2). Nov 06 22:33:33.329 WARN unable to fetch block #00000000476781c04b82b3ea91af1a86f3a863e1c9312b50302ffa01b7bdf960: will retry in a few seconds (attempt #2).

@lgalabru, do you run a testnet ordhook instance? I see mainnet archive file: https://archive.hiro.so/mainnet/ordhook/ but I don't see one for testnet.

diwakergupta commented 1 year ago

@lgalabru, do you run a testnet ordhook instance? I see mainnet archive file: https://archive.hiro.so/mainnet/ordhook/ but I don't see one for testnet.

We don't currently run ordhook or Ordinals API on Bitcoin primary testnet.

lgalabru commented 11 months ago

Testnet is now supported thanks to #208

github-actions[bot] commented 11 months ago

:tada: This issue has been resolved in version 1.1.1 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket:

hirosystems / ordhook

"internal error: entered unreachable code" when starting service on testnet #187