blockchain-etl / bitcoin-etl

ETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ
https://twitter.com/BlockchainETL
MIT License
411 stars 123 forks source link

I got ValueError('"result" is None in the JSON RPC response {}. Request: {}', resp_item.get('error'), text) #18

Closed rascal-3 closed 5 years ago

rascal-3 commented 5 years ago

Hi there,

Although I succeeded bittcoinetl get_block_range_for_date command, I got following errors and failed when I run bitcoinetl export_all command.

$ bitcoinetl export_all -s 0 -e 573154 -b 100 -p http://bitcoin:local321@127.0.0.1:8332 -c bitcoin
2019-04-25 12:03:05,537 - export_all [INFO] - Exporting blocks 00000000-00000099 to output/blocks/start_block=00000000/end_block=00000099/blocks_00000000_00000099.json
2019-04-25 12:03:05,537 - export_all [INFO] - Exporting transactions from blocks 00000000-00000099 to output/transactions/start_block=00000000/end_block=00000099/transactions_00000000_00000099.json
2019-04-25 12:03:05,537 - ProgressLogger [INFO] - Started work. Items to process: 100.
Traceback (most recent call last):
  File "/home/rascal/.local/lib/python3.6/site-packages/blockchainetl/jobs/base_job.py", line 28, in run
    self._export()
  File "/home/rascal/.local/lib/python3.6/site-packages/bitcoinetl/jobs/export_blocks_job.py", line 68, in _export
    total_items=self.end_block - self.start_block + 1
  File "/home/rascal/.local/lib/python3.6/site-packages/blockchainetl/executors/batch_work_executor.py", line 48, in execute
    self.executor.submit(self._fail_safe_execute, work_handler, batch)
  File "/home/rascal/.local/lib/python3.6/site-packages/blockchainetl/executors/fail_safe_executor.py", line 31, in submit
    self._check_completed_futures()
  File "/home/rascal/.local/lib/python3.6/site-packages/blockchainetl/executors/fail_safe_executor.py", line 47, in _check_completed_futures
    future.result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/rascal/.local/lib/python3.6/site-packages/blockchainetl/executors/batch_work_executor.py", line 53, in _fail_safe_execute
    work_handler(batch)
  File "/home/rascal/.local/lib/python3.6/site-packages/bitcoinetl/jobs/export_blocks_job.py", line 72, in _export_batch
    blocks = self.btc_service.get_blocks(block_number_batch, self.export_transactions)
  File "/home/rascal/.local/lib/python3.6/site-packages/bitcoinetl/service/btc_service.py", line 59, in get_blocks
    return self.get_blocks_by_hashes(block_hashes, with_transactions)
  File "/home/rascal/.local/lib/python3.6/site-packages/bitcoinetl/service/btc_service.py", line 67, in get_blocks_by_hashes
    block_detail_response = self.bitcoin_rpc.batch(block_detail_rpc)
  File "/home/rascal/.local/lib/python3.6/site-packages/bitcoinetl/rpc/bitcoin_rpc.py", line 53, in batch
    raise ValueError('"result" is None in the JSON RPC response {}. Request: {}', resp_item.get('error'), text)
ValueError: ('"result" is None in the JSON RPC response {}. Request: {}', {'code': -1, 'message': 'JSON value is not a boolean as expected'}, '[{"jsonrpc": "2.0", "method": "getblock", "params": ["00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048", 2], "id": "1"}]')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rascal/.local/bin/bitcoinetl", line 11, in <module>
    sys.exit(cli())
  File "/home/rascal/.local/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/rascal/.local/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/rascal/.local/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/rascal/.local/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/rascal/.local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/rascal/.local/lib/python3.6/site-packages/bitcoinetl/cli/export_all.py", line 101, in export_all
    output_dir, provider_uri, max_workers, export_batch_size)
  File "/home/rascal/.local/lib/python3.6/site-packages/bitcoinetl/jobs/export_all.py", line 98, in export_all
    job.run()
  File "/home/rascal/.local/lib/python3.6/site-packages/blockchainetl/jobs/base_job.py", line 30, in run
    self._end()
  File "/home/rascal/.local/lib/python3.6/site-packages/bitcoinetl/jobs/export_blocks_job.py", line 85, in _end
    self.batch_work_executor.shutdown()
  File "/home/rascal/.local/lib/python3.6/site-packages/blockchainetl/executors/batch_work_executor.py", line 68, in shutdown
    self.executor.shutdown()
  File "/home/rascal/.local/lib/python3.6/site-packages/blockchainetl/executors/fail_safe_executor.py", line 39, in shutdown
    self._check_completed_futures()
  File "/home/rascal/.local/lib/python3.6/site-packages/blockchainetl/executors/fail_safe_executor.py", line 47, in _check_completed_futures
    future.result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/rascal/.local/lib/python3.6/site-packages/blockchainetl/executors/batch_work_executor.py", line 53, in _fail_safe_execute
    work_handler(batch)
  File "/home/rascal/.local/lib/python3.6/site-packages/bitcoinetl/jobs/export_blocks_job.py", line 72, in _export_batch
    blocks = self.btc_service.get_blocks(block_number_batch, self.export_transactions)
  File "/home/rascal/.local/lib/python3.6/site-packages/bitcoinetl/service/btc_service.py", line 59, in get_blocks
    return self.get_blocks_by_hashes(block_hashes, with_transactions)
  File "/home/rascal/.local/lib/python3.6/site-packages/bitcoinetl/service/btc_service.py", line 67, in get_blocks_by_hashes
    block_detail_response = self.bitcoin_rpc.batch(block_detail_rpc)
  File "/home/rascal/.local/lib/python3.6/site-packages/bitcoinetl/rpc/bitcoin_rpc.py", line 53, in batch
    raise ValueError('"result" is None in the JSON RPC response {}. Request: {}', resp_item.get('error'), text)
ValueError: ('"result" is None in the JSON RPC response {}. Request: {}', {'code': -1, 'message': 'JSON value is not a boolean as expected'}, '[{"jsonrpc": "2.0", "method": "getblock", "params": ["000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f", 2], "id": "1"}]')

I use Ubuntu 18.04 LTS, Python version is 3.6.8, and I got result by running bitcoin-cli getblockchaininfo

$ bitcoin-cli -rpcuser=bitcoin -rpcpassword=local321 getblockchaininfo
{
  "chain": "main",
  "blocks": 573170,
  "headers": 573170,
  "bestblockhash": "0000000000000000000411ca1744b51606c9f222b5df1ac4a6b566e7bcf6ce57",
  "difficulty": 6353030562983.983,
  "mediantime": 1556191378,
  "verificationprogress": 0.9999995394033131,
  "chainwork": "000000000000000000000000000000000000000005f6153034b35a5d4cb02cbf",
  "pruned": false,
  "softforks": [
    {
      "id": "bip34",
      "version": 2,
      "enforce": {
        "status": true,
        "found": 1000,
        "required": 750,
        "window": 1000
      },
      "reject": {
        "status": true,
        "found": 1000,
        "required": 950,
        "window": 1000
      }
    },
    {
      "id": "bip66",
      "version": 3,
      "enforce": {
        "status": true,
        "found": 1000,
        "required": 750,
        "window": 1000
      },
      "reject": {
        "status": true,
        "found": 1000,
        "required": 950,
        "window": 1000
      }
    },
    {
      "id": "bip65",
      "version": 4,
      "enforce": {
        "status": true,
        "found": 1000,
        "required": 750,
        "window": 1000
      },
      "reject": {
        "status": true,
        "found": 1000,
        "required": 950,
        "window": 1000
      }
    }
  ],
  "bip9_softforks": [
    {
      "id": "csv",
      "status": "active"
    }
  ]
}

ethreum-etl is working well, by the way...

Thank you,

medvedev1088 commented 5 years ago

For some reason the bitcoin rpc expects a boolean value for the 2nd param in this request:

[{"jsonrpc": "2.0", "method": "getblock", "params": ["00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048", 2], "id": "1"}]

Try running this request with curl and pass true instead of 2 and see if it works. Are you sure it's a bitcoin node and not e.g. bitcoin cash?

rascal-3 commented 5 years ago

@medvedev1088 Thanks for your replying.

I got this:

$ curl --user bitcoin --data-binary '{"jsonrpc": "2.0", "method": "getblock", "params": ["00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048", true], "id": "1"}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
Enter host password for user 'bitcoin':
{"result":{"hash":"00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048","confirmations":573307,"size":215,"height":1,"version":1,"merkleroot":"0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098","tx":["0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098"],"time":1231469665,"mediantime":1231469665,"nonce":2573394689,"bits":"1d00ffff","difficulty":1,"chainwork":"0000000000000000000000000000000000000000000000000000000200020002","previousblockhash":"000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f","nextblockhash":"000000006a625f06636b8bb6ac7b960a8d03705d1ace08b1a19da3fdcc99ddbd"},"error":null,"id":"1"}

According to the blockhash, it was Bitcoin Cash... I installed 4 full nodes(Ethereum, Bitcoin, Litecoin, Bitcoin Cash), but I might have overwritten btcfullnode by bchfullnode...

I'm gonna check these again.

Thanks for your help!!

rascal-3 commented 5 years ago

Now I can extract data! Thanks a lot!!