Timeouts when running the Build script

[davidnovotech]

Block 317777 crawled Block 317778 crawled Traceback (most recent call last): File "build_cluster.py", line 52, in start() File "build_cluster.py", line 17, in start while builder.crawl_block(block_id): File "/opt/Bitcluster/crawler/base_crawler.py", line 27, in crawl_block block = self.proxy.getblock(hash) File "/home/apollo/.local/share/virtualenvs/Bitcluster/lib/python3.4/site-packages/bitcoin/rpc.py", line 382, in getblock r = self._call('getblock', block_hash, False) File "/home/apollo/.local/share/virtualenvs/Bitcluster/lib/python3.4/site-packages/bitcoin/rpc.py", line 186, in _call response = self._get_response() File "/home/apollo/.local/share/virtualenvs/Bitcluster/lib/python3.4/site-packages/bitcoin/rpc.py", line 207, in _get_response http_response = self.__conn.getresponse() File "/usr/lib/python3.4/http/client.py", line 1172, in getresponse response.begin() File "/usr/lib/python3.4/http/client.py", line 351, in begin version, status, reason = self._read_status() File "/usr/lib/python3.4/http/client.py", line 313, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib/python3.4/socket.py", line 371, in readinto return self._sock.recv_into(b) socket.timeout: timed out

This is probably due to some kind of network connectivity issue. Is there a known way to improve reliability ? Like an auto retry or a longer wait time.

Can we as well restart the Build script up to the last block crawled or it needs to start from 1 every time ? Will that affect the mapping afterwards ?

Also can we get explanations on whats in a block exactly ? I suppose :

• Addresses
• Time stamps
• Wallet ID ? ? what else

[mathieulavoie]

You can restart from where it have failed for the build_cluster.py script. To improve relability, make sure that your bitcoind deamon did finish to download the full blockchain (see the debug.log file) before starting the build_cluster script. I'll try to reproduce your issue.

To get explanation on how the data is kept in the database, you can refer to this wiki page

[davidnovotech]

Thank you for checking. Please keep me posted.

*I believe the bitcoin daemon finished the download. I have no output when I start it. What is the message I should see in debug that will tell me if its done ?

[davidnovotech]

2016-08-10 15:12:55 UpdateTip: new best=000000000000000000da1631a8d7ce980248dc9300f4d1648d5d27f50cb77cbf height=424576 log2_work=85.11187 tx=148297686 date=2016-08-10 15:14:55 progress=1.000001 cache=13.6MiB(8109tx)

I receive those every now and then. Does that means the daemon is always downloading new blocks ?

[davidnovotech]

Were you able to reproduce ?

[mathieulavoie]

I wasn't able to reproduce. However I did implement an auto-reconnect mechanisms that will be included in the next commit.

[davidnovotech]

Thank you