search

hirosystems / ordhook

Build indexers, standards and protocols on top of Ordinals and Inscriptions (BRC20, etc).
Apache License 2.0
182 stars 54 forks source link

spin up ordhook as a servcie , occur error ''database disk image is malformed" #172

Closed Hodgewhj closed 12 months ago

Hodgewhj commented 1 year ago

when spinning up ordhook as a service ,use below command: ordhook service start --config-path=./Ordhook.toml

image
lgalabru commented 1 year ago

Ha, strange, there could be a regression. Could you try 

ordhook db new --config-path ...

before starting your service?

MikaelBTC commented 1 year ago

Ha, strange, there could be a regression. Could you try

ordhook db new --config-path ...

before starting your service?

Hello, we have had the same problem too. But not fixed after using: ordhook db new before start the service.

tag: v1.0.0.

Do you have any other way to work around or fix this? Thanks.

lgalabru commented 1 year ago

This issue should be fixed in the latest develop + main, and the CI is preparing a new release v1.0.1

MikaelBTC commented 1 year ago

This issue should be fixed in the latest develop + main, and the CI is preparing a new release v1.0.1

Thank you very much, we will try it again.

Hodgewhj commented 1 year ago

Ha, strange, there could be a regression. Could you try

ordhook db new --config-path ...

before starting your service?

Hello, we have had the same problem too. But not fixed after using: ordhook db new before start the service.

tag: v1.0.0.

Do you have any other way to work around or fix this? Thanks.

i init ordhook db before staring it, but still has some problems

image
Hodgewhj commented 1 year ago

This issue should be fixed in the latest develop + main, and the CI is preparing a new release v1.0.1

seems don't have release v1.0.1

image
Hodgewhj commented 1 year ago

Ha, strange, there could be a regression. Could you try

ordhook db new --config-path ...

before starting your service?

Hello, we have had the same problem too. But not fixed after using: ordhook db new before start the service. tag: v1.0.0. Do you have any other way to work around or fix this? Thanks.

i init ordhook db before staring it, but still has some problems image

Hodgewhj commented 1 year ago

Ha, strange, there could be a regression. Could you try

ordhook db new --config-path ...

before starting your service?

Hello, we have had the same problem too. But not fixed after using: ordhook db new before start the service.

tag: v1.0.0.

Do you have any other way to work around or fix this? Thanks.

seems thats i can spin up the stream service, but data is not complete

image
MikaelBTC commented 1 year ago

This issue should be fixed in the latest develop + main, and the CI is preparing a new release v1.0.1

Thank you very much, we will try it again.

After we tried again, the logs show it loaded all the blocks but showed another error when we restart it(for many times).

Sep 15 07:16:04.023 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:04.025 INFO Inscription 3a6e0e0304d33faa199858997477fb61e45dc7125d4c5f398a6f2e2e87af5be2i0 moved from 3a6e0e0304d33faa199858997477fb61e45dc7125d4c5f398a6f2e2e87af5be2:0:0 to f923569c504ef268292a2f275155c8f13c2737732985ec59b00966ccaa1bdc54:0 (block: 775550) Sep 15 07:16:04.156 INFO Block #775550 processed, revealed 2 inscriptions [0, 0] and 2 transfers Sep 15 07:16:04.156 WARN Dropping updates for block #775550, activities present in database Sep 15 07:16:04.166 INFO Inscription ee7ef87345b6be19df4154f621ea5944518b7155d01d4784b2086c1e06e2539di0 moved from ee7ef87345b6be19df4154f621ea5944518b7155d01d4784b2086c1e06e2539d:0:0 to c4181b9e5b5251ffe617fab53a9cb886c2aedaa093cbb5b52c7b24a94816353d:0 (block: 775551) Sep 15 07:16:04.173 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:04.173 INFO Inscription dbef456d715407c5f5abec5ea4956c46968ef90b2043d9b77ab909e0aa6ca122i0 moved from dbef456d715407c5f5abec5ea4956c46968ef90b2043d9b77ab909e0aa6ca122:0:0 to 16dc5d326c9053fcf3c2650e3c39e3050f21ba920406ce529007b36562b7074f:0 (block: 775551) Sep 15 07:16:04.173 INFO Inscription 3d192df0a8e6f5a21886cdb0c34670d5173e6abe24b18ef6dccb61e15b5b1f1bi0 moved from 3d192df0a8e6f5a21886cdb0c34670d5173e6abe24b18ef6dccb61e15b5b1f1b:0:0 to bd5248a0e8058260f7575a6f51057953416cf33ac7ac9a87f26b3054cc244c57:0 (block: 775551) Sep 15 07:16:04.174 INFO Inscription 096d063fe17f20ece54e6876b0e25a5c1942b7167e9d7d8a414f87f304d8d526i0 moved from 096d063fe17f20ece54e6876b0e25a5c1942b7167e9d7d8a414f87f304d8d526:0:0 to dd0a00975ff7643e7ecfd714a0785205a5d21efab73141ede1ff4a8662202f65:0 (block: 775551) Sep 15 07:16:04.177 INFO Inscription 157542fb1bfc9da46a98478f441bc7c070f85c1166ae8b13175e71cdd5540645i0 moved from 157542fb1bfc9da46a98478f441bc7c070f85c1166ae8b13175e71cdd5540645:0:0 to 6757c2eed9041f0d6d76f3e95fab41778e08631e3c0ac54f4e9a399605a0207a:0 (block: 775551) Sep 15 07:16:04.281 INFO Inscription 0ea90d4856b7406b47d9200860931dee45d5ab56c8bd516999100bc70598c059i0 moved from 967a16bff82a7ccd03d73069cab3b643ec2c5d18a5175284b1da5ed4762e78f7:0:634206065 to aca4a57303c70268905ac5ef3df57f5a7316b6b0e79741718c23955bca91f66b:109 (block: 775551) Sep 15 07:16:04.354 INFO Block #775551 processed, revealed 26 inscriptions [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] and 6 transfers Sep 15 07:16:04.354 WARN Dropping updates for block #775551, activities present in database Sep 15 07:16:04.415 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:04.512 INFO Inscription 3ab98ce137268fe1a3bd9902bbb2b8f07a653a8d7a65a36908274be09cb01568i0 moved from 3ab98ce137268fe1a3bd9902bbb2b8f07a653a8d7a65a36908274be09cb01568:0:0 to 26de6022bdf4d203c8d13e92fed6677cfbca3eb88659412c62026287a77095d5:0 (block: 775552) Sep 15 07:16:04.522 INFO Block #775552 processed, revealed 45 inscriptions [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] and 1 transfers Sep 15 07:16:04.534 WARN Dropping updates for block #775552, activities present in database Sep 15 07:16:04.541 INFO Inscription 146578101bf92a799324f2838e897d4956b15cbe9957bdd5ee9ebdbc5d24b186i0 moved from 8dc9bd25fb33256f2182ece6d5466c8421883041e32897fe9bd3abdb8ab7c605:4:2442259858 to 381f23115bffa3b7459439c5ea8e038eb95bb5ebc89bdd7d66720fe2fbea9084:4 (block: 775553) Sep 15 07:16:04.580 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:04.584 INFO Inscription f3a5d3edd2dcc57471906ce68bc42226669dc42346d21fc09bea740fb1dfd036i0 moved from f3a5d3edd2dcc57471906ce68bc42226669dc42346d21fc09bea740fb1dfd036:0:0 to d3130f356dde5de5220fbc1641df26f8eca438eee2e14eee833f75209911b478:0 (block: 775553) Sep 15 07:16:04.586 INFO Inscription a8980015638bca3c6416f25ee13d24b9a1c3f2a7372c87cdd76a9ac8e3b26f38i0 moved from a8980015638bca3c6416f25ee13d24b9a1c3f2a7372c87cdd76a9ac8e3b26f38:0:0 to 3e90bac40982a71869e18eba39698229c154197f0cecd3d0be71402dab0e9e8e:0 (block: 775553) Sep 15 07:16:04.586 INFO Inscription 3d138e21dea8a1a15a8981d0be921bcea5cfd4a5a4f02076db048b217665e04bi0 moved from 3d138e21dea8a1a15a8981d0be921bcea5cfd4a5a4f02076db048b217665e04b:0:0 to 938a0705d06534145b533203010f43fb3789b0440ab7718e12c457a9b785bbbe:0 (block: 775553) Sep 15 07:16:04.704 INFO Block #775553 processed, revealed 225 inscriptions [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] and 4 transfers Sep 15 07:16:04.704 WARN Dropping updates for block #775553, activities present in database Sep 15 07:16:04.704 INFO Number of inscriptions in block #775554 to process: 1 (L1 cache hits: 0, queue: [], L1 cache len: 0, L2 cache len: 0) Sep 15 07:16:04.773 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:05.144 INFO Completed ordinal number retrieval for Satpoint 0x2ada24450bb5d23c5b91f1ee30a6ce378aaa9d74555476f7505093fd8959a741:0:0 (block: #178145:3487091423, transfers: 334, progress: 1/1, priority queue: true, thread: 0) Sep 15 07:16:05.147 ERRO Unable to retrieve cached inscription data for inscription 0x77d31c95c5902a9528fd059d84244b985dde06b2843a80fda8d703215e4febfe Sep 15 07:16:05.147 ERRO Unable to retrieve cached inscription data for inscription 0x6f3b992c34c51ca06f58ea916266618aaffbf70bf2f079c8d29d9761eddfeced Sep 15 07:16:05.147 ERRO Unable to retrieve cached inscription data for inscription 0xc0ebbf93aba271b2faad85ea58471a479865b54e79537f69f8739bfa3aceb3d5 Sep 15 07:16:05.151 INFO Satoshi #890728487091423 was inscribed with blessed inscription c0ebbf93aba271b2faad85ea58471a479865b54e79537f69f8739bfa3aceb3d5i0, cursing inscription 2ada24450bb5d23c5b91f1ee30a6ce378aaa9d74555476f7505093fd8959a741i0 Sep 15 07:16:05.151 INFO Inscription 2ada24450bb5d23c5b91f1ee30a6ce378aaa9d74555476f7505093fd8959a741i0 (#-1) detected on Satoshi 890728487091423 (block #775554, 334 transfers) Sep 15 07:16:05.151 ERRO Unable to retrieve cached inscription data for inscription 0x9c7635f496387a18d94cd05cf09ed0535e78e6f2aa7822c501fee769a4d39518 Sep 15 07:16:05.152 ERRO UNIQUE constraint failed: inscriptions.inscription_id thread 'Inscription indexing runloop' panicked at 'index out of bounds: the len is 1 but the index is 1', components/ordhook-core/src/db/mod.rs:1060:24 stack backtrace: 0: rust_begin_unwind at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/panicking.rs:593:5 1: core::panicking::panic_fmt at /builddir/build/BUILD/rustc-1.72.0-src/library/core/src/panicking.rs:67:14 2: core::panicking::panic_bounds_check at /builddir/build/BUILD/rustc-1.72.0-src/library/core/src/panicking.rs:162:5 3: ordhook::db::parse_satpoint_to_watch 4: ordhook::db::insert_inscription_in_locations 5: ordhook::db::update_inscriptions_with_block 6: ordhook::core::protocol::inscription_sequencing::augment_block_with_ordinals_inscriptions_data_and_write_to_db_tx 7: ordhook::core::pipeline::processors::inscription_indexing::process_block 8: ordhook::core::pipeline::processors::inscription_indexing::process_blocks

error

Hodgewhj commented 1 year ago

Ha, strange, there could be a regression. Could you try

ordhook db new --config-path ...

before starting your service?

Hello, we have had the same problem too. But not fixed after using: ordhook db new before start the service.

tag: v1.0.0.

Do you have any other way to work around or fix this? Thanks.

i use "ordhook db sync --config-path" command, but i can only sync block height to 200000

image

This issue should be fixed in the latest develop + main, and the CI is preparing a new release v1.0.1

Thank you very much, we will try it again.

After we tried again, the logs show it loaded all the blocks but showed another error when we restart it(for many times).

Sep 15 07:16:04.023 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:04.025 INFO Inscription 3a6e0e0304d33faa199858997477fb61e45dc7125d4c5f398a6f2e2e87af5be2i0 moved from 3a6e0e0304d33faa199858997477fb61e45dc7125d4c5f398a6f2e2e87af5be2:0:0 to f923569c504ef268292a2f275155c8f13c2737732985ec59b00966ccaa1bdc54:0 (block: 775550) Sep 15 07:16:04.156 INFO Block #775550 processed, revealed 2 inscriptions [0, 0] and 2 transfers Sep 15 07:16:04.156 WARN Dropping updates for block #775550, activities present in database Sep 15 07:16:04.166 INFO Inscription ee7ef87345b6be19df4154f621ea5944518b7155d01d4784b2086c1e06e2539di0 moved from ee7ef87345b6be19df4154f621ea5944518b7155d01d4784b2086c1e06e2539d:0:0 to c4181b9e5b5251ffe617fab53a9cb886c2aedaa093cbb5b52c7b24a94816353d:0 (block: 775551) Sep 15 07:16:04.173 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:04.173 INFO Inscription dbef456d715407c5f5abec5ea4956c46968ef90b2043d9b77ab909e0aa6ca122i0 moved from dbef456d715407c5f5abec5ea4956c46968ef90b2043d9b77ab909e0aa6ca122:0:0 to 16dc5d326c9053fcf3c2650e3c39e3050f21ba920406ce529007b36562b7074f:0 (block: 775551) Sep 15 07:16:04.173 INFO Inscription 3d192df0a8e6f5a21886cdb0c34670d5173e6abe24b18ef6dccb61e15b5b1f1bi0 moved from 3d192df0a8e6f5a21886cdb0c34670d5173e6abe24b18ef6dccb61e15b5b1f1b:0:0 to bd5248a0e8058260f7575a6f51057953416cf33ac7ac9a87f26b3054cc244c57:0 (block: 775551) Sep 15 07:16:04.174 INFO Inscription 096d063fe17f20ece54e6876b0e25a5c1942b7167e9d7d8a414f87f304d8d526i0 moved from 096d063fe17f20ece54e6876b0e25a5c1942b7167e9d7d8a414f87f304d8d526:0:0 to dd0a00975ff7643e7ecfd714a0785205a5d21efab73141ede1ff4a8662202f65:0 (block: 775551) Sep 15 07:16:04.177 INFO Inscription 157542fb1bfc9da46a98478f441bc7c070f85c1166ae8b13175e71cdd5540645i0 moved from 157542fb1bfc9da46a98478f441bc7c070f85c1166ae8b13175e71cdd5540645:0:0 to 6757c2eed9041f0d6d76f3e95fab41778e08631e3c0ac54f4e9a399605a0207a:0 (block: 775551) Sep 15 07:16:04.281 INFO Inscription 0ea90d4856b7406b47d9200860931dee45d5ab56c8bd516999100bc70598c059i0 moved from 967a16bff82a7ccd03d73069cab3b643ec2c5d18a5175284b1da5ed4762e78f7:0:634206065 to aca4a57303c70268905ac5ef3df57f5a7316b6b0e79741718c23955bca91f66b:109 (block: 775551) Sep 15 07:16:04.354 INFO Block #775551 processed, revealed 26 inscriptions [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] and 6 transfers Sep 15 07:16:04.354 WARN Dropping updates for block #775551, activities present in database Sep 15 07:16:04.415 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:04.512 INFO Inscription 3ab98ce137268fe1a3bd9902bbb2b8f07a653a8d7a65a36908274be09cb01568i0 moved from 3ab98ce137268fe1a3bd9902bbb2b8f07a653a8d7a65a36908274be09cb01568:0:0 to 26de6022bdf4d203c8d13e92fed6677cfbca3eb88659412c62026287a77095d5:0 (block: 775552) Sep 15 07:16:04.522 INFO Block #775552 processed, revealed 45 inscriptions [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] and 1 transfers Sep 15 07:16:04.534 WARN Dropping updates for block #775552, activities present in database Sep 15 07:16:04.541 INFO Inscription 146578101bf92a799324f2838e897d4956b15cbe9957bdd5ee9ebdbc5d24b186i0 moved from 8dc9bd25fb33256f2182ece6d5466c8421883041e32897fe9bd3abdb8ab7c605:4:2442259858 to 381f23115bffa3b7459439c5ea8e038eb95bb5ebc89bdd7d66720fe2fbea9084:4 (block: 775553) Sep 15 07:16:04.580 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:04.584 INFO Inscription f3a5d3edd2dcc57471906ce68bc42226669dc42346d21fc09bea740fb1dfd036i0 moved from f3a5d3edd2dcc57471906ce68bc42226669dc42346d21fc09bea740fb1dfd036:0:0 to d3130f356dde5de5220fbc1641df26f8eca438eee2e14eee833f75209911b478:0 (block: 775553) Sep 15 07:16:04.586 INFO Inscription a8980015638bca3c6416f25ee13d24b9a1c3f2a7372c87cdd76a9ac8e3b26f38i0 moved from a8980015638bca3c6416f25ee13d24b9a1c3f2a7372c87cdd76a9ac8e3b26f38:0:0 to 3e90bac40982a71869e18eba39698229c154197f0cecd3d0be71402dab0e9e8e:0 (block: 775553) Sep 15 07:16:04.586 INFO Inscription 3d138e21dea8a1a15a8981d0be921bcea5cfd4a5a4f02076db048b217665e04bi0 moved from 3d138e21dea8a1a15a8981d0be921bcea5cfd4a5a4f02076db048b217665e04b:0:0 to 938a0705d06534145b533203010f43fb3789b0440ab7718e12c457a9b785bbbe:0 (block: 775553) Sep 15 07:16:04.704 INFO Block #775553 processed, revealed 225 inscriptions [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] and 4 transfers Sep 15 07:16:04.704 WARN Dropping updates for block #775553, activities present in database Sep 15 07:16:04.704 INFO Number of inscriptions in block #775554 to process: 1 (L1 cache hits: 0, queue: [], L1 cache len: 0, L2 cache len: 0) Sep 15 07:16:04.773 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:05.144 INFO Completed ordinal number retrieval for Satpoint 0x2ada24450bb5d23c5b91f1ee30a6ce378aaa9d74555476f7505093fd8959a741:0:0 (block: #178145:3487091423, transfers: 334, progress: 1/1, priority queue: true, thread: 0) Sep 15 07:16:05.147 ERRO Unable to retrieve cached inscription data for inscription 0x77d31c95c5902a9528fd059d84244b985dde06b2843a80fda8d703215e4febfe Sep 15 07:16:05.147 ERRO Unable to retrieve cached inscription data for inscription 0x6f3b992c34c51ca06f58ea916266618aaffbf70bf2f079c8d29d9761eddfeced Sep 15 07:16:05.147 ERRO Unable to retrieve cached inscription data for inscription 0xc0ebbf93aba271b2faad85ea58471a479865b54e79537f69f8739bfa3aceb3d5 Sep 15 07:16:05.151 INFO Satoshi #890728487091423 was inscribed with blessed inscription c0ebbf93aba271b2faad85ea58471a479865b54e79537f69f8739bfa3aceb3d5i0, cursing inscription 2ada24450bb5d23c5b91f1ee30a6ce378aaa9d74555476f7505093fd8959a741i0 Sep 15 07:16:05.151 INFO Inscription 2ada24450bb5d23c5b91f1ee30a6ce378aaa9d74555476f7505093fd8959a741i0 (#-1) detected on Satoshi 890728487091423 (block #775554, 334 transfers) Sep 15 07:16:05.151 ERRO Unable to retrieve cached inscription data for inscription 0x9c7635f496387a18d94cd05cf09ed0535e78e6f2aa7822c501fee769a4d39518 Sep 15 07:16:05.152 ERRO UNIQUE constraint failed: inscriptions.inscription_id thread 'Inscription indexing runloop' panicked at 'index out of bounds: the len is 1 but the index is 1', components/ordhook-core/src/db/mod.rs:1060:24 stack backtrace: 0: rust_begin_unwind at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/panicking.rs:593:5 1: core::panicking::panic_fmt at /builddir/build/BUILD/rustc-1.72.0-src/library/core/src/panicking.rs:67:14 2: core::panicking::panic_bounds_check at /builddir/build/BUILD/rustc-1.72.0-src/library/core/src/panicking.rs:162:5 3: ordhook::db::parse_satpoint_to_watch 4: ordhook::db::insert_inscription_in_locations 5: ordhook::db::update_inscriptions_with_block 6: ordhook::core::protocol::inscription_sequencing::augment_block_with_ordinals_inscriptions_data_and_write_to_db_tx 7: ordhook::core::pipeline::processors::inscription_indexing::process_block 8: ordhook::core::pipeline::processors::inscription_indexing::process_blocks

error

i can now syn block heigh to 200000, how did you load all the blocks

image
MikaelBTC commented 1 year ago

Ha, strange, there could be a regression. Could you try

ordhook db new --config-path ...

before starting your service?

Hello, we have had the same problem too. But not fixed after using: ordhook db new before start the service. tag: v1.0.0. Do you have any other way to work around or fix this? Thanks.

i use "ordhook db sync --config-path" command, but i can only sync block height to 200000 image

This issue should be fixed in the latest develop + main, and the CI is preparing a new release v1.0.1

Thank you very much, we will try it again.

After we tried again, the logs show it loaded all the blocks but showed another error when we restart it(for many times). Sep 15 07:16:04.023 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:04.025 INFO Inscription 3a6e0e0304d33faa199858997477fb61e45dc7125d4c5f398a6f2e2e87af5be2i0 moved from 3a6e0e0304d33faa199858997477fb61e45dc7125d4c5f398a6f2e2e87af5be2:0:0 to f923569c504ef268292a2f275155c8f13c2737732985ec59b00966ccaa1bdc54:0 (block: 775550) Sep 15 07:16:04.156 INFO Block #775550 processed, revealed 2 inscriptions [0, 0] and 2 transfers Sep 15 07:16:04.156 WARN Dropping updates for block #775550, activities present in database Sep 15 07:16:04.166 INFO Inscription ee7ef87345b6be19df4154f621ea5944518b7155d01d4784b2086c1e06e2539di0 moved from ee7ef87345b6be19df4154f621ea5944518b7155d01d4784b2086c1e06e2539d:0:0 to c4181b9e5b5251ffe617fab53a9cb886c2aedaa093cbb5b52c7b24a94816353d:0 (block: 775551) Sep 15 07:16:04.173 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:04.173 INFO Inscription dbef456d715407c5f5abec5ea4956c46968ef90b2043d9b77ab909e0aa6ca122i0 moved from dbef456d715407c5f5abec5ea4956c46968ef90b2043d9b77ab909e0aa6ca122:0:0 to 16dc5d326c9053fcf3c2650e3c39e3050f21ba920406ce529007b36562b7074f:0 (block: 775551) Sep 15 07:16:04.173 INFO Inscription 3d192df0a8e6f5a21886cdb0c34670d5173e6abe24b18ef6dccb61e15b5b1f1bi0 moved from 3d192df0a8e6f5a21886cdb0c34670d5173e6abe24b18ef6dccb61e15b5b1f1b:0:0 to bd5248a0e8058260f7575a6f51057953416cf33ac7ac9a87f26b3054cc244c57:0 (block: 775551) Sep 15 07:16:04.174 INFO Inscription 096d063fe17f20ece54e6876b0e25a5c1942b7167e9d7d8a414f87f304d8d526i0 moved from 096d063fe17f20ece54e6876b0e25a5c1942b7167e9d7d8a414f87f304d8d526:0:0 to dd0a00975ff7643e7ecfd714a0785205a5d21efab73141ede1ff4a8662202f65:0 (block: 775551) Sep 15 07:16:04.177 INFO Inscription 157542fb1bfc9da46a98478f441bc7c070f85c1166ae8b13175e71cdd5540645i0 moved from 157542fb1bfc9da46a98478f441bc7c070f85c1166ae8b13175e71cdd5540645:0:0 to 6757c2eed9041f0d6d76f3e95fab41778e08631e3c0ac54f4e9a399605a0207a:0 (block: 775551) Sep 15 07:16:04.281 INFO Inscription 0ea90d4856b7406b47d9200860931dee45d5ab56c8bd516999100bc70598c059i0 moved from 967a16bff82a7ccd03d73069cab3b643ec2c5d18a5175284b1da5ed4762e78f7:0:634206065 to aca4a57303c70268905ac5ef3df57f5a7316b6b0e79741718c23955bca91f66b:109 (block: 775551) Sep 15 07:16:04.354 INFO Block #775551 processed, revealed 26 inscriptions [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] and 6 transfers Sep 15 07:16:04.354 WARN Dropping updates for block #775551, activities present in database Sep 15 07:16:04.415 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:04.512 INFO Inscription 3ab98ce137268fe1a3bd9902bbb2b8f07a653a8d7a65a36908274be09cb01568i0 moved from 3ab98ce137268fe1a3bd9902bbb2b8f07a653a8d7a65a36908274be09cb01568:0:0 to 26de6022bdf4d203c8d13e92fed6677cfbca3eb88659412c62026287a77095d5:0 (block: 775552) Sep 15 07:16:04.522 INFO Block #775552 processed, revealed 45 inscriptions [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] and 1 transfers Sep 15 07:16:04.534 WARN Dropping updates for block #775552, activities present in database Sep 15 07:16:04.541 INFO Inscription 146578101bf92a799324f2838e897d4956b15cbe9957bdd5ee9ebdbc5d24b186i0 moved from 8dc9bd25fb33256f2182ece6d5466c8421883041e32897fe9bd3abdb8ab7c605:4:2442259858 to 381f23115bffa3b7459439c5ea8e038eb95bb5ebc89bdd7d66720fe2fbea9084:4 (block: 775553) Sep 15 07:16:04.580 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:04.584 INFO Inscription f3a5d3edd2dcc57471906ce68bc42226669dc42346d21fc09bea740fb1dfd036i0 moved from f3a5d3edd2dcc57471906ce68bc42226669dc42346d21fc09bea740fb1dfd036:0:0 to d3130f356dde5de5220fbc1641df26f8eca438eee2e14eee833f75209911b478:0 (block: 775553) Sep 15 07:16:04.586 INFO Inscription a8980015638bca3c6416f25ee13d24b9a1c3f2a7372c87cdd76a9ac8e3b26f38i0 moved from a8980015638bca3c6416f25ee13d24b9a1c3f2a7372c87cdd76a9ac8e3b26f38:0:0 to 3e90bac40982a71869e18eba39698229c154197f0cecd3d0be71402dab0e9e8e:0 (block: 775553) Sep 15 07:16:04.586 INFO Inscription 3d138e21dea8a1a15a8981d0be921bcea5cfd4a5a4f02076db048b217665e04bi0 moved from 3d138e21dea8a1a15a8981d0be921bcea5cfd4a5a4f02076db048b217665e04b:0:0 to 938a0705d06534145b533203010f43fb3789b0440ab7718e12c457a9b785bbbe:0 (block: 775553) Sep 15 07:16:04.704 INFO Block #775553 processed, revealed 225 inscriptions [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] and 4 transfers Sep 15 07:16:04.704 WARN Dropping updates for block #775553, activities present in database Sep 15 07:16:04.704 INFO Number of inscriptions in block #775554 to process: 1 (L1 cache hits: 0, queue: [], L1 cache len: 0, L2 cache len: 0) Sep 15 07:16:04.773 INFO Trigger http://192.168.1.191:8800/ordinals/events successful Sep 15 07:16:05.144 INFO Completed ordinal number retrieval for Satpoint 0x2ada24450bb5d23c5b91f1ee30a6ce378aaa9d74555476f7505093fd8959a741:0:0 (block: #178145:3487091423, transfers: 334, progress: 1/1, priority queue: true, thread: 0) Sep 15 07:16:05.147 ERRO Unable to retrieve cached inscription data for inscription 0x77d31c95c5902a9528fd059d84244b985dde06b2843a80fda8d703215e4febfe Sep 15 07:16:05.147 ERRO Unable to retrieve cached inscription data for inscription 0x6f3b992c34c51ca06f58ea916266618aaffbf70bf2f079c8d29d9761eddfeced Sep 15 07:16:05.147 ERRO Unable to retrieve cached inscription data for inscription 0xc0ebbf93aba271b2faad85ea58471a479865b54e79537f69f8739bfa3aceb3d5 Sep 15 07:16:05.151 INFO Satoshi #890728487091423 was inscribed with blessed inscription c0ebbf93aba271b2faad85ea58471a479865b54e79537f69f8739bfa3aceb3d5i0, cursing inscription 2ada24450bb5d23c5b91f1ee30a6ce378aaa9d74555476f7505093fd8959a741i0 Sep 15 07:16:05.151 INFO Inscription 2ada24450bb5d23c5b91f1ee30a6ce378aaa9d74555476f7505093fd8959a741i0 (#-1) detected on Satoshi 890728487091423 (block #775554, 334 transfers) Sep 15 07:16:05.151 ERRO Unable to retrieve cached inscription data for inscription 0x9c7635f496387a18d94cd05cf09ed0535e78e6f2aa7822c501fee769a4d39518 Sep 15 07:16:05.152 ERRO UNIQUE constraint failed: inscriptions.inscription_id thread 'Inscription indexing runloop' panicked at 'index out of bounds: the len is 1 but the index is 1', components/ordhook-core/src/db/mod.rs:1060:24 stack backtrace: 0: rust_begin_unwind at /builddir/build/BUILD/rustc-1.72.0-src/library/std/src/panicking.rs:593:5 1: core::panicking::panic_fmt at /builddir/build/BUILD/rustc-1.72.0-src/library/core/src/panicking.rs:67:14 2: core::panicking::panic_bounds_check at /builddir/build/BUILD/rustc-1.72.0-src/library/core/src/panicking.rs:162:5 3: ordhook::db::parse_satpoint_to_watch 4: ordhook::db::insert_inscription_in_locations 5: ordhook::db::update_inscriptions_with_block 6: ordhook::core::protocol::inscription_sequencing::augment_block_with_ordinals_inscriptions_data_and_write_to_db_tx 7: ordhook::core::pipeline::processors::inscription_indexing::process_block 8: ordhook::core::pipeline::processors::inscription_indexing::process_blocks error

i can now syn block heigh to 200000, how did you load all the blocks

image

Just start the service and wait for 10 hours about to load, did not do anything else: ordhook service start --post-to=http://192.168.1.191:8800/ordinals/events --config-path=./Ordhook.toml

lgalabru commented 1 year ago

@MikaelBTC @Hodgewhj thank you both for giving this a shot!

@Hodgewhj: your issue should be fixed in the latest develop, can you checkout the source and build the project on your machine? Our CI needs a bit of love, I'm trying to kick off a new v1.0.1 build.

@MikaelBTC: I'm surprised that you were able to pass the issue that @MikaelBTC was experiencing. Would you mind re-installing, re-generate a new config, and give it another try?

The indexing is composed of 2 parts:

https://ordinals.hiro.so is powered by this software however we use some backups for speeding up the process. It looks like we have not ran these code paths (that now should be fixed) for some time, I just started an index from scratch on my machine to give it a shot.

MikaelBTC commented 1 year ago

@MikaelBTC @Hodgewhj thank you both for giving this a shot!

@Hodgewhj: your issue should be fixed in the latest develop, can you checkout the source and build the project on your machine? Our CI needs a bit of love, I'm trying to kick off a new v1.0.1 build.

@MikaelBTC: I'm surprised that you were able to pass the issue that @MikaelBTC was experiencing. Would you mind re-installing, re-generate a new config, and give it another try?

The indexing is composed of 2 parts:

  • Step 1: Download bitcoin blocks from bitcoind and compress them: depending on your setup, it can take 6h to 2-3 days. Ideally, bitcoind and ordhook are living on the same machine. This step will produce hord.rocksdb, a 50Gb db helping computing ordinals number as fast as possible. This part is intensive on disk (SSD / NVMe for the win) + CPU + networking if bitcoind is remote.
  • Step 2: Compute inscriptions metadata: basically parse blocks and transactions, and compute ordinals numbers, inscriptions numbers, etc. Depending on the performance of your machine, this step could take half a day to 2-3 days. This step will produce hord.sqlite including all the metadata and locations for all the inscriptions.

https://ordinals.hiro.so is powered by this software however we use some backups for speeding up the process. It looks like we have not ran these code paths (that now should be fixed) for some time, I just started an index from scratch on my machine to give it a shot.

I think because I have ran the scan firstly. Once I have data on the hord.sqlite, then I can process the data to 774410, not the latest block.

MikaelBTC commented 1 year ago

@MikaelBTC @Hodgewhj thank you both for giving this a shot!

@Hodgewhj: your issue should be fixed in the latest develop, can you checkout the source and build the project on your machine? Our CI needs a bit of love, I'm trying to kick off a new v1.0.1 build.

@MikaelBTC: I'm surprised that you were able to pass the issue that @MikaelBTC was experiencing. Would you mind re-installing, re-generate a new config, and give it another try?

The indexing is composed of 2 parts:

  • Step 1: Download bitcoin blocks from bitcoind and compress them: depending on your setup, it can take 6h to 2-3 days. Ideally, bitcoind and ordhook are living on the same machine. This step will produce hord.rocksdb, a 50Gb db helping computing ordinals number as fast as possible. This part is intensive on disk (SSD / NVMe for the win) + CPU + networking if bitcoind is remote.
  • Step 2: Compute inscriptions metadata: basically parse blocks and transactions, and compute ordinals numbers, inscriptions numbers, etc. Depending on the performance of your machine, this step could take half a day to 2-3 days. This step will produce hord.sqlite including all the metadata and locations for all the inscriptions.

https://ordinals.hiro.so is powered by this software however we use some backups for speeding up the process. It looks like we have not ran these code paths (that now should be fixed) for some time, I just started an index from scratch on my machine to give it a shot.

Thanks for your support too. I can not run the ordhook service successfully. Also I am not sure how to connect ordhook with ordinals-api, If you can show some instruments I will be very appreciate.

lgalabru commented 1 year ago

I think because I have ran the scan firstly. Once I have data on the hord.sqlite, then I can process the data to 774410, not the latest block.

Ah yes good catch. I'm going to push a patch to cover this use case. Have you tried downloading the backups mentioned above?

Also I am not sure how to connect ordhook with ordinals-api, If you can show some instruments I will be very appreciate.

This is something that @ryanwaits and @mefrem could probably help you with.

Hodgewhj commented 1 year ago

I think because I have ran the scan firstly. Once I have data on the hord.sqlite, then I can process the data to 774410, not the latest block.

Ah yes good catch. I'm going to push a patch to cover this use case. Have you tried downloading the backups mentioned above?

Also I am not sure how to connect ordhook with ordinals-api, If you can show some instruments I will be very appreciate.

This is something that @ryanwaits and @mefrem could probably help you with.

okay thanks , i will try according to your suggestion. and by the way, could i know how long it will take if i write a project to parse the ordianls and trace the satoshi transfer and so on.

MikaelBTC commented 1 year ago

I think because I have ran the scan firstly. Once I have data on the hord.sqlite, then I can process the data to 774410, not the latest block.

Ah yes good catch. I'm going to push a patch to cover this use case. Have you tried downloading the backups mentioned above?

Also I am not sure how to connect ordhook with ordinals-api, If you can show some instruments I will be very appreciate.

This is something that @ryanwaits and @mefrem could probably help you with.

Thanks @lgalabru.

I started the service and then logs showed processing data. I did not download the backup manually and the service did not download it either. It takes me more than 3 days to finish the init. But I am facing a new problem. After ordhook service finished the init, it showed "Waiting for ZMQ messages from bitcoind" but did not process the latest block.

lgalabru commented 12 months ago

Waiting for ZMQ messages from bitcoind

Is your bitcoind configured to publish zmq message, on the port that ordhook is listening to?

With the latest release, ordhook should be able to gracefully do what it needs to do if it previously downloaded hord.sqlite after a scan and needs to run in service mode.

MikaelBTC commented 12 months ago

Waiting for ZMQ messages from bitcoind

Is your bitcoind configured to publish zmq message, on the port that ordhook is listening to?

With the latest release, ordhook should be able to gracefully do what it needs to do if it previously downloaded hord.sqlite after a scan and needs to run in service mode.

Hi lgalabru, I think I fixed this, everything looks good now. Thanks for your amazing work. My bitcoind was started with zmq ip address configured, but in Ordhook.xml I use 127.0.0.1 to connect it, so did not get the latest block. It is better you can give some logs showing the zmq can not be connected.

lgalabru commented 12 months ago

Perfect, and thank you for the kind words. Feel free to open issues during your experience with ordhook when you spot room for improvements!