search

swapper-org / NodeChain

Nodechain is a service that allows you to create nodes on a blockchain and connect to them natively through APIs. The goal is to allow the user to build their own nodes natively without having to rely on external services.
MIT License
7 stars 5 forks source link

ElectRS service stops after 4 failed attempts #86

Closed phoenix-web3 closed 2 years ago

phoenix-web3 commented 2 years ago

Issue

/build/entrypoint.sh: setting script as command
/build/entrypoint.sh: setting data directory to /build/.electrsConfig { log: StdErrLog { verbosity: Warn, quiet: false, show_level: true, timestamp: Millisecond, modules: [], writer: "stderr", color_choice: Never }, network_type: Testnet, db_path: "/build/.electrs/testnet", daemon_dir: "/home/bitcoin/.bitcoin/testnet3", blocks_dir: "/home/bitcoin/.bitcoin/testnet3/blocks", dae
mon_rpc_addr: 172.18.0.2:8332, cookie: Some("-----:----"), electrum_rpc_addr: 0.0.0.0:60001, http_addr: 127.0.0.1:3001, http_socket_file: None, monitoring_addr: 127.0.0.1:14224, jsonrpc_import: false, light_mode: false, address_search: false, index_unspendables: false, cors: None, precache_scripts: None, utxos
_limit: 500, electrum_txs_limit: 500, electrum_banner: "Welcome to electrs-esplora 0.4.1" }
2022-02-09T15:05:11.095+00:00 - WARN - reconnecting to bitcoind: {"code":-28,"message":"Loading block index..."}
2022-02-09T15:05:14.095+00:00 - WARN - reconnecting to bitcoind: {"code":-28,"message":"Loading block index..."}
2022-02-09T15:05:17.096+00:00 - WARN - reconnecting to bitcoind: {"code":-28,"message":"Loading block index..."}
2022-02-09T15:05:20.096+00:00 - WARN - reconnecting to bitcoind: {"code":-28,"message":"Loading block index..."}
phoenix-web3 commented 2 years ago

Background

Reproduced steps

~/projects/swapper-org/NodeChain/scripts$ docker logs --follow btctestnetapi_electrs_1 
/build/entrypoint.sh: assuming arguments for electrs
/build/entrypoint.sh: setting script as command
/build/entrypoint.sh: setting data directory to /build/.electrs

Config { log: StdErrLog { verbosity: Warn, quiet: false, show_level: true, timestamp: Millisecond, modules: [], writer: "stderr", color_choice: Never }, network_type: Testnet, db_path: "/build/.electrs/testnet", daemon_dir: "/home/bitcoin/.bitcoin/testnet3", blocks_dir: "/home/bitcoin/.bitcoin/testnet3/blocks", daemon_rpc_addr: 172.20.0.2:8332, cookie: Some("----:----"), electrum_rpc_addr: 0.0.0.0:60001, http_addr: 127.0.0.1:3001, http_socket_file: None, monitoring_addr: 127.0.0.1:14224, jsonrpc_import: false, light_mode: false, address_search: false, index_unspendables: false, cors: None, precache_scripts: None, utxos_limit: 500, electrum_txs_limit: 500, electrum_banner: "Welcome to electrs-esplora 0.4.1" }
2022-02-10T00:57:01.043+00:00 - WARN - waiting for bitcoind sync to finish: 0/0 blocks, verification progress: 0.000%
2022-02-10T00:57:06.073+00:00 - WARN - waiting for bitcoind sync to finish: 0/104000 blocks, verification progress: 0.000%
2022-02-10T00:57:11.075+00:00 - WARN - waiting for bitcoind sync to finish: 0/196000 blocks, verification progress: 0.000%
2022-02-10T00:57:16.088+00:00 - WARN - waiting for bitcoind sync to finish: 0/290000 blocks, verification progress: 0.000%
2022-02-10T00:57:21.088+00:00 - WARN - waiting for bitcoind sync to finish: 0/380000 blocks, verification progress: 0.000%
2022-02-10T00:57:26.113+00:00 - WARN - waiting for bitcoind sync to finish: 0/474000 blocks, verification progress: 0.000%
2022-02-10T00:57:31.114+00:00 - WARN - waiting for bitcoind sync to finish: 0/568000 blocks, verification progress: 0.000%
2022-02-10T00:57:36.115+00:00 - WARN - waiting for bitcoind sync to finish: 0/664000 blocks, verification progress: 0.000%
2022-02-10T00:57:41.144+00:00 - WARN - waiting for bitcoind sync to finish: 0/760000 blocks, verification progress: 0.000%
2022-02-10T00:57:46.145+00:00 - WARN - waiting for bitcoind sync to finish: 0/852000 blocks, verification progress: 0.000%
2022-02-10T00:57:51.145+00:00 - WARN - waiting for bitcoind sync to finish: 0/944000 blocks, verification progress: 0.000%
2022-02-10T00:57:57.097+00:00 - WARN - waiting for bitcoind sync to finish: 0/1036000 blocks, verification progress: 0.000%
2022-02-10T00:58:02.098+00:00 - WARN - waiting for bitcoind sync to finish: 0/1130000 blocks, verification progress: 0.000%
2022-02-10T00:58:07.132+00:00 - WARN - waiting for bitcoind sync to finish: 0/1224000 blocks, verification progress: 0.000%
2022-02-10T00:58:12.133+00:00 - WARN - waiting for bitcoind sync to finish: 0/1316000 blocks, verification progress: 0.000%
2022-02-10T00:58:17.134+00:00 - WARN - waiting for bitcoind sync to finish: 0/1408000 blocks, verification progress: 0.000%
2022-02-10T00:58:22.154+00:00 - WARN - waiting for bitcoind sync to finish: 0/1498000 blocks, verification progress: 0.000%
2022-02-10T00:58:27.155+00:00 - WARN - waiting for bitcoind sync to finish: 0/1590000 blocks, verification progress: 0.000%
2022-02-10T00:58:32.156+00:00 - WARN - waiting for bitcoind sync to finish: 0/1682000 blocks, verification progress: 0.000%
2022-02-10T00:58:37.157+00:00 - WARN - waiting for bitcoind sync to finish: 0/1774000 blocks, verification progress: 0.000%
2022-02-10T00:58:42.158+00:00 - WARN - waiting for bitcoind sync to finish: 0/1862000 blocks, verification progress: 0.000%
2022-02-10T00:58:47.159+00:00 - WARN - waiting for bitcoind sync to finish: 720/1956000 blocks, verification progress: 0.003%
2022-02-10T00:58:52.160+00:00 - WARN - waiting for bitcoind sync to finish: 1472/2048000 blocks, verification progress: 0.004%
2022-02-10T00:58:57.179+00:00 - WARN - waiting for bitcoind sync to finish: 2192/2140000 blocks, verification progress: 0.005%
2022-02-10T00:59:02.180+00:00 - WARN - waiting for bitcoind sync to finish: 7926/2140444 blocks, verification progress: 0.030%
2022-02-10T00:59:07.183+00:00 - WARN - waiting for bitcoind sync to finish: 15044/2140444 blocks, verification progress: 0.041%
2022-02-10T00:59:12.185+00:00 - WARN - waiting for bitcoind sync to finish: 16902/2140444 blocks, verification progress: 0.044%
2022-02-10T00:59:17.186+00:00 - WARN - waiting for bitcoind sync to finish: 17926/2140444 blocks, verification progress: 0.046%
2022-02-10T00:59:22.188+00:00 - WARN - waiting for bitcoind sync to finish: 18948/2140444 blocks, verification progress: 0.048%
phoenix-web3 commented 2 years ago

Can you pass the whole trace @bridgedragon ? My local is going good

bridgedragon commented 2 years ago

It is the whole trace 😂

bridgedragon commented 2 years ago

To reproduce the issue:

  1. Sync the entire blockchain
  2. Restart the nodes

The problem goes when electrs is starting while bitcoin core is loading block index. Loading block index only takes time when blockchain has enough information (fully synced, for example).

I do not see an inmediate solution for this problem, because bitcoincore does not signal anything when block index is loaded, and electrs does not have any option to retry the connection.

The only thing that goes through my head is instructing electrs to wait a defined amount of seconds after the container starts, but it is an ugly solution and doesn't guarantee fully functionality in any scenario.

Other approaches are writing some kind of script at electrs entrypoint, checking periodically that bitcoincore is started and syncing, but it could be a pain in the a*

bridgedragon commented 2 years ago

Or, maybe, making a PR in electrs, because it shouldn't have this behavior.

phoenix-web3 commented 2 years ago

What about to restart the docker service (eletrs) if fails? This should be a quick hotfix until we get a solution @bridgedragon

bridgedragon commented 2 years ago

It is the thing to do in deployment, my instance is working with that approach. But it is important and must be fixed in a future.