`socket.timeout: timed out` (v9.49.1+)

[mtbitcoin]

This is a on a windows box

Status: Running v9.49.2 of counterpartyd. Status: Connecting to backend. Status: Checking version. Status: Acquiring lock. Status: Connecting to database. Status: Connecting to block explorer. Status: Resuming parsing. ERROR: Traceback (most recent call last): File "C:\counterpartyd_build\dist\counterpartyd\lib\blocks.py", line 977, in follow mempool_tx_index = list_tx(db, None, block_index, curr_time, tx_hash, mempool_tx_index ) File "C:\counterpartyd_build\dist\counterpartyd\lib\blocks.py", line 653, in list_tx source, destination, btc_amount, fee, data = get_tx_info(tx_dict['hex'], block_index) File "C:\counterpartyd_build\dist\counterpartyd\lib\blocks.py", line 341, in get_tx_info

tx_info = get_tx_info2(tx_hex, block_parser=block_parser)

File "C:\counterpartyd_build\dist\counterpartyd\lib\blocks.py", line 560, in get_tx_info 2 vin_ctx = proxy.getrawtransaction(vin.prevout.hash) File "C:\counterpartyd_build\env\lib\site-packages\bitcoin\rpc.py", line 307, in getrawt ransaction r = self._call('getrawtransaction', b2lx(txid), 1 if verbose else 0) File "C:\counterpartyd_build\env\lib\site-packages\bitcoin\rpc.py", line 144, in _call response = self._get_response() File "C:\counterpartyd_build\env\lib\site-packages\bitcoin\rpc.py", line 179, in _get_re sponse http_response = self.__conn.getresponse() File "C:\Python34\Lib\http\client.py", line 1172, in getresponse response.begin() File "C:\Python34\Lib\http\client.py", line 351, in begin version, status, reason = self._read_status() File "C:\Python34\Lib\http\client.py", line 313, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "C:\Python34\Lib\socket.py", line 371, in readinto return self._sock.recv_into(b) socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\counterpartyd_build\dist\counterpartyd\counterpartyd.py", line 956, in

blocks.follow(db)

File "C:\counterpartyd_build\dist\counterpartyd\lib\blocks.py", line 978, in follow except util.BitcoindError: AttributeError: 'module' object has no attribute 'BitcoindError'

[ouziel-slama]

@adamkrellenstein, I'am really not sure! I'am fighting with this issue from yesterday, and with or without the batch RPC call, I have a lot of timeout errors, even with a simple get_asset_info calls.. Maybe we should make batch requests by chunks of 10 or 20 queries max, but I'am really not sure that is the root of the problem. Also when counterpartyd return a timeout error, I can not make a clean stop with ctrl-c and I need to use kill, I think it's related to theses RPC calls stucked somewhere..

[adamkrellenstein]

@ouziel-slama, huh, even with the new timeout of an hour and python-bitcoinlib 0.3.0 (which makes the timeout actually work)?

[ouziel-slama]

I'am trying to make work with counterblockd/counterwallet, and there is timeout in the browser and in counterblockd.. and somewhere I don't know.. because I always get a timeout after 1mn.. but in any case it's impossible to let in the counterwallet/counterblockd a timeout of 1mn, there will be too many simultaneous queries, this can be only a temporary solution I think.

[mtbitcoin]

I am still running into issues even with 0.3.0, the latest develop pull and/or @faizkhan00's fix. The counterpartyd can even be brought down by an external bitcoin RPC call (another program calling the bitcoind api)

[ghost]

I ran it down all the way to the socket.py (which is part of the stdlib) last night, its definitely something in the C layer thats causing it, normal HTTP requests are fine but it looks like nonstandard protocols are throwing it for a loop. I experimented with wierd/removed headers and also got somewhat mixed results. Might look at this again tonight

[ghost]

best case could be to bring the issue upstream to the stdlib guys, and then perhaps use a workaround using the json-rpc library in the meanwhile (note: have not tested this as a fix yet)

[flaneur2020]

the Timeout issue in syncing blockchain disappeared after applied @faizkhan00 's patch, running counterpartyd server and it synced block to 335622 (the current block height), but it raised ServerError: Counterparty database is behind backend. on executing get_debits.

[adamkrellenstein]

@Fleurer. Good, and that's supposed to happen.

[flaneur2020]

@adamkrellenstein 335622 is the current block height of the block chain, would the API works on next block get synced ?

[adamkrellenstein]

Oh, I misread the number. That's strange. Run bitcoind getinfo and paste the output.

[flaneur2020]

[deleted] move to https://github.com/CounterpartyXCP/counterpartyd/issues/559

[adamkrellenstein]

Also, run the server with --verbose, and let's move the discussion here: https://github.com/CounterpartyXCP/counterpartyd/issues/559

[flaneur2020]

@adamkrellenstein ok, but it seems that counterpartyd doesn't have a --verbose flag ?

mcp@mcounterparty:/tmp/python-bitcoinlib-python-bitcoinlib-v0.3.0$ counterpartyd server --verbose
usage: counterpartyd [-h] [-V] [-v] [--testnet] [--testcoin] [--unconfirmed]
                     [--encoding ENCODING] [--fee-per-kb FEE_PER_KB]
                     [--regular-dust-size REGULAR_DUST_SIZE]
                     [--multisig-dust-size MULTISIG_DUST_SIZE]
                     [--op-return-value OP_RETURN_VALUE] [--unsigned]
                     [--data-dir DATA_DIR] [--database-file DATABASE_FILE]
                     [--config-file CONFIG_FILE] [--log-file LOG_FILE]
                     [--backend-rpc-connect BACKEND_RPC_CONNECT]
                     [--backend-rpc-port BACKEND_RPC_PORT]
                     [--backend-rpc-user BACKEND_RPC_USER]
                     [--backend-rpc-password BACKEND_RPC_PASSWORD]
                     [--backend-rpc-ssl] [--backend-rpc-ssl-verify]
                     [--backend-poll-interval BACKEND_POLL_INTERVAL]
                     [--blockchain-service-name BLOCKCHAIN_SERVICE_NAME]
                     [--blockchain-service-connect BLOCKCHAIN_SERVICE_CONNECT]
                     [--rpc-host RPC_HOST] [--rpc-port RPC_PORT]
                     [--rpc-user RPC_USER] [--rpc-password RPC_PASSWORD]
                     [--rpc-allow-cors]
                     {server,send,order,btcpay,issuance,broadcast,bet,dividend,burn,cancel,rps,rpsresolve,publish,execute,reparse,rollback,kickstart}
                     ...
counterpartyd: error: unrecognized arguments: --verbose

[adamkrellenstein]

Responded at #559.

[flaneur2020]

@adamkrellenstein the ServerError: Counterparty database is behind backend issue seems disappeared after restarted bitcoind & counterpartyd, thank you

[flaneur2020]

unfortunately there are still Timeout issue, but it comes from the bitcoind with the addrindex patch.

when counterpartyd is up, every command of bitcoind like bitcoind getblockhash 352332 or bitcoind getinfo would block forever; the bitcoind commands became available as soon as counterpartyd shutdown.

it seems there are some operations from counterpartyd blocked bitcoind completely.

update: the official bitcoind also get blocked forever after counterpartyd is up, and becomes available as soon as counterpartyd is down.

[mtbitcoin]

Just to add that I also had the " ServerError: Counterparty database is behind backend" after applying @faizkhan00's patch but it would still timeout.

I am also running bitcoind with the addrindex patch . But the issues only start after ? 9.49.1

[adamkrellenstein]

Would people try seeing if the latest commit to develop (https://github.com/CounterpartyXCP/counterpartyd/commit/2caa11cc419b4eaeea5ba6b83941aa2ae208ed2f) has any effect on the socket.timeout error?

[adamkrellenstein]

Re: the "database behind backend" error, 9c184d924219f95459213e3534f6352a1426e921 fixed a long-standing issue where the database wasn't being checked to see if it was up-to-date, so yes, that error message is new. If it is appearing where it legitimately should not be, please create a new Issue, as it's unrelated to the socket.timeout error in question here.

[mtbitcoin]

the ""database behind backend"" only appeared after applying the custom patch suggested by @faizkhan00 to fix the timeout issue. After I removed the patch this was not an issue. I suspect his modified rpc calls where getting interpreted differently. I am currently running your ( 2caa11c ), but I am waiting for a new block to be processed first to see if it still holds

[flaneur2020]

I reproduced the "database behinde backend" issue

[adamkrellenstein]

OK, for now, try not running faiz's patch, which though it may help, probably only does so accidentally.

[mtbitcoin]

@adamkrellenstein The latest develop commit works. So far no more timeouts

[flaneur2020]

@adamkrellenstein thank you, the latest develop branch works and no "database behind backend",

here the latest exception: ServerError: name 'proxy' is not defined

[mtbitcoin]

No more timeout issues but a new error while reparsing the transactions . I should probably open a new issue?

---

2014-12-24-T06:43:11Coordinated Universal Time [INFO] Running v9.49.2 of counterpartyd. 2014-12-24-T06:43:11Coordinated Universal Time [INFO] Connecting to backend. 2014-12-24-T06:43:11Coordinated Universal Time [INFO] Checking version. 2014-12-24-T06:43:12Coordinated Universal Time [INFO] Acquiring lock. 2014-12-24-T06:43:12Coordinated Universal Time [INFO] Connecting to database. 2014-12-24-T06:43:36Coordinated Universal Time [INFO] Connecting to block explorer. 2014-12-24-T06:43:38Coordinated Universal Time [INFO] Resuming parsing. 2014-12-24-T06:43:39Coordinated Universal Time [WARNING] Blockchain reorganisation at block 335644. 2014-12-24-T06:43:39Coordinated Universal Time [WARNING] Reparsing all transactions. 2014-12-24-T06:43:43Coordinated Universal Time [ERROR] Unhandled Exception Traceback (most recent call last): File "C:\counterpartyd_build\dist\counterpartyd\counterpartyd.py", line 931, in blocks.follow(db, proxy) File "C:\counterpartyd_build\dist\counterpartyd\lib\blocks.py", line 914, in follow reparse(db, block_index=current_index-1, quiet=True) File "C:\counterpartyd_build\dist\counterpartyd\lib\blocks.py", line 645, in reparse log.setLevel(logging.WARNING) AttributeError: 'module' object has no attribute 'setLevel'

[adamkrellenstein]

@mtbitcoin. Glad to hear it! That AttributeError is a totally separate issue (handled at #560), thankfully. There was just a recent reorg. which triggered it.

@Fleurer, that ServerError, however, is related. Can you give me the full traceback if/when you see that again?