Use Bitcoin regtest

[oblak-be]

Dear maintainers/contributors

Bitcoinlib version 0.6.13

Goal:

Use Bitcoin regtest for platform code testing having the availability to provide test users with credit and run our platform tests.

Problem:

When opening an existing wallet, the wallet.scan() function fails with error: bitcoinlib.services.services.ServiceError: No successful response from any serviceprovider: ['bitcoind']

Steps taken:

• add to providers.json:

"bitcoind": {
    "provider": "bitcoind",
    "network": "regtest",
    "client_class": "BitcoindClient()",
    "provider_coin_id": "",
    "url": "http://<user>:<pass>@bitcoin:8332",
    "api_key": "",
    "priority": 10,
    "denominator": 100000000,
    "network_overrides": null
},

• Change prefix_bech32 in network.jon for regtest:

  "regtest":
  {
    "description": "Bitcoin regtest",
    "currency_name": "regtest",
    "currency_name_plural": "bitcoins",
    "currency_symbol": "r฿",
    "currency_code": "rBTC",
    "prefix_address": "00",
    "prefix_address_p2sh": "05",
    "prefix_bech32": "bcrt",
    "prefix_wif": "80",
    "prefixes_wif": [
      ["0488B21E", "xpub", "public",  false, "legacy",      "p2pkh"],
      ["0488B21E", "xpub", "public",  true,  "legacy",      "p2sh"],
      ["0488ADE4", "xprv", "private", false, "legacy",      "p2pkh"],
      ["0488ADE4", "xprv", "private", true,  "legacy",      "p2sh"],
      ["049D7CB2", "ypub", "public",  false, "p2sh-segwit", "p2sh_p2wpkh"],
      ["0295B43F", "Ypub", "public",  true,  "p2sh-segwit", "p2sh_p2wsh"],
      ["049D7878", "yprv", "private", false, "p2sh-segwit", "p2sh_p2wpkh"],
      ["0295B005", "Yprv", "private", true,  "p2sh-segwit", "p2sh_p2wsh"],
      ["04B24746", "zpub", "public",  false, "segwit",      "p2wpkh"],
      ["02AA7ED3", "Zpub", "public",  true,  "segwit",      "p2wsh"],
      ["04B2430C", "zprv", "private", false, "segwit",      "p2wpkh"],
      ["02AA7A99", "Zprv", "private", true,  "segwit",      "p2wsh"]
    ],
    "bip44_cointype": 0,
    "denominator": 0.00000001,
    "dust_amount": 1000,
    "fee_default": null,
    "fee_min": 1000,
    "fee_max": 1000000,
    "priority": 0
  },

• have a functinoing bitcoin-core node running in regtest mode on url from providers.json

• create some wallets using bitcoinlib

• mine some coins on bitcoin node and send some to a bitcoinlib wallet using bitcoin-cli on bitcoin node.

• bitcoinlib commands:

  
  >>> wallet = bitcoinlib.wallets.wallet_create_or_open("test-user")

bitcoinlib.services.bitcoind.BitcoindClient().network

wallet.network bitcoinlib.services.bitcoind.BitcoindClient().gettransactions(wallet_address) [ , ", line 1, in File "/data/.local/lib/python3.8/site-packages/bitcoinlib/wallets.py", line 1849, in scan self.transactions_update_confirmations() File "/data/.local/lib/python3.8/site-packages/bitcoinlib/wallets.py", line 3021, in transactions_update_confirmations srv = Service(network=network, providers=self.providers, cache_uri=self.db_cache_uri) File "/data/.local/lib/python3.8/site-packages/bitcoinlib/services/services.py", line 145, in __init__ self._blockcount = self.blockcount() File "/data/.local/lib/python3.8/site-packages/bitcoinlib/services/services.py", line 495, in blockcount new_count = self._provider_execute('blockcount') File "/data/.local/lib/python3.8/site-packages/bitcoinlib/services/services.py", line 224, in _provider_execute raise ServiceError("No successful response from any serviceprovider: %s" % list(self.providers.keys())) bitcoinlib.services.services.ServiceError: No successful response from any serviceprovider: ['bitcoind'] # set network to bitcoin for test wallet.network = bitcoinlib.networks.Network("bitcoin") w.scan() Traceback (most recent call last): File "", line 1, in File "/data/.local/lib/python3.8/site-packages/bitcoinlib/wallets.py", line 1877, in scan if self.scan_key(key): File "/data/.local/lib/python3.8/site-packages/bitcoinlib/wallets.py", line 1800, in scan_key n_new = self.transactions_update(key_id=key.key_id) File "/data/.local/lib/python3.8/site-packages/bitcoinlib/wallets.py", line 3102, in transactions_update srv = Service(network=network, providers=self.providers, cache_uri=self.db_cache_uri) File "/data/.local/lib/python3.8/site-packages/bitcoinlib/services/services.py", line 145, in __init__ self._blockcount = self.blockcount() File "/data/.local/lib/python3.8/site-packages/bitcoinlib/services/services.py", line 495, in blockcount new_count = self._provider_execute('blockcount') File "/data/.local/lib/python3.8/site-packages/bitcoinlib/services/services.py", line 224, in _provider_execute raise ServiceError("No successful response from any serviceprovider: %s" % list(self.providers.keys())) bitcoinlib.services.services.ServiceError: No successful response from any serviceprovider: ['bitcoind'] # scan() hangs now, but throws similar error ``` Maybe I'm doing something wrong or have missed a configuration step?

Any help appreciated and thanks for the great project!

[mccwdev]

It seems related to this: https://github.com/1200wd/bitcoinlib/commit/c6efbdeb137c9850ca5a10b01ed1b48e26589f79 fixed with this pull request: https://github.com/1200wd/bitcoinlib/pull/352

[n1rna]

I'm not sure if the mentioned PR is addressing all the issues mentioned in the issue. I'm also facing the issue with the wrong chain param for the regtest network. The prefix_bech32 value must be bcrt @mccwdev

https://github.com/bitcoin/bitcoin/blob/master/src/kernel/chainparams.cpp#L522

[oblak-be]

After some more testing and digging i found out that

When wallet.scan() is called in our case, BitcoindClient.gettransactions(wallet_address, '', 20) is called and fails

error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data/.local/lib/python3.8/site-packages/bitcoinlib/services/bitcoind.py", line 223, in gettransactions
    raise ClientError("Address %s not found in bitcoind wallet, use 'importpubkey' or 'importaddress' to add "
bitcoinlib.services.baseclient.ClientError: Address bcrt1qn... not found in bitcoind wallet, use 'importpubkey' or 'importaddress' to add address to wallet.

When changing res['iswatchonly'] = True wallet.scan() works and the balance is updated.

[n1rna]

My workaround for now is pointing to a custom data directory as follows:

[locations]
# Location of BitcoinLib data, configuration and log files. Relative paths will be based in installation directory
data_dir=./data

In the BCL_CONFIG_FILE and then fixing the networks.json file for my own use case.

P.S. You need to replicate all the required files in the data directory: https://github.com/1200wd/bitcoinlib/tree/master/bitcoinlib/data