search

CounterpartyXCP / counterblock

Provides extended API services to Counterwallet, as well as Counterparty 3rd-party applications
http://counterparty.io
MIT License
16 stars 68 forks source link

Infinite redirect loop on block 348078 #115

Closed ArtemKo7v closed 9 years ago

ArtemKo7v commented 9 years ago

Installed a new Federated Node instance using these instructions: http://counterparty.io/docs/federated_node/ Counterparty parsed blockchain till the last block. But Counterblock went into infinite redirect loop on block 348078:

root@oaz-fnode:/home/xcp/.cache# tail -n 30 counterblock/log/server.log
2015-03-22-T05:28:45+0000:INFO:siofeeds:Starting up socket.io server (block event feed)...
2015-03-22-T05:28:45+0000:INFO:siofeeds:Starting up socket.io server (chat feed)...
2015-03-22-T05:28:45+0000:INFO:startup:Starting up counterparty block feed poller...
2015-03-22-T05:28:45+0000:WARNING:startup:Support email setting not set: To enable, please specify an email for the 'support-email' setting in your counterblockd.conf
2015-03-22-T05:28:45+0000:INFO:startup:Starting up RPC API handler...
2015-03-22-T05:28:45+0000:WARNING:database:Pruning to block 348078 ...
2015-03-22-T05:28:48+0000:INFO:server:counterblock Version 1.1.0 starting ...
2015-03-22-T05:28:48+0000:INFO:startup:Loading counterwallet config at '/home/xcp/counterwallet/counterwallet.conf.json'
2015-03-22-T05:28:48+0000:INFO:startup:Checking/updating GeoIP.dat ...
2015-03-22-T05:28:48+0000:INFO:startup:GeoIP.dat database up to date. Not downloading.
2015-03-22-T05:28:48+0000:INFO:database:Connecting to mongoDB backend ...
2015-03-22-T05:28:48+0000:INFO:cache:Connecting to redis @ 127.0.0.1
2015-03-22-T05:28:48+0000:INFO:siofeeds:Starting up socket.io server (block event feed)...

The same redirect loop on Testnet on block 324178:

root@oaz-fnode:/home/xcp/.cache# tail -n 30 counterblock/log/server.testnet.log
2015-03-23-T07:15:14+0000:WARNING:startup:Support email setting not set: To enable, please specify an email for the 'support-email' setting in your counterblockd.conf
2015-03-23-T07:15:14+0000:INFO:startup:Starting up RPC API handler...
2015-03-23-T07:15:14+0000:WARNING:database:Pruning to block 324178 ...
2015-03-23-T07:15:14+0000:INFO:tasks:Revised wallet statistics for partial day 2015-1-9: {'referers': [], 'network': 'testnet', 'login_count': 3, 'distinct_login_count': 2, 'new_count': 0}
2015-03-23-T07:15:16+0000:INFO:server:counterblock Version 1.1.0 starting ...
2015-03-23-T07:15:16+0000:INFO:startup:Loading counterwallet config at '/home/xcp/counterwallet/counterwallet.conf.json'
2015-03-23-T07:15:16+0000:INFO:startup:Checking/updating GeoIP.dat ...
2015-03-23-T07:15:16+0000:INFO:startup:GeoIP.dat database up to date. Not downloading.
2015-03-23-T07:15:16+0000:INFO:database:Connecting to mongoDB backend ...
2015-03-23-T07:15:16+0000:INFO:cache:Connecting to redis @ 127.0.0.1
2015-03-23-T07:15:16+0000:INFO:siofeeds:Starting up socket.io server (block event feed)...
2015-03-23-T07:15:16+0000:INFO:siofeeds:Starting up socket.io server (chat feed)...
2015-03-23-T07:15:16+0000:INFO:startup:Starting up counterparty block feed poller...
robby-d commented 9 years ago

Please launch via the console (via the command in the federated node troubleshooting section) and paste the exception you get. Also, please make sure you're running the latest commit from master, as I thought I fixed this last week (unless it is another similar issue)

ArtemKo7v commented 9 years ago 
WARNING:database: Pruning to block 348078 ...
Traceback (most recent call last):
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/gevent/greenlet.py", line 327, in run
    result = self._run(*self.args, **self.kwargs)
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/blockfeed.py", line 198, in process_cp_blockfeed
    cp_running_info['version_major'], cp_running_info['version_minor']))
TypeError: not all arguments converted during string formatting
<Greenlet at 0x7f2934681410: process_cp_blockfeed(<zmq.green.core._Socket object at 0x7f29346cf870>)> failed with TypeError

Traceback (most recent call last):
  File "/usr/local/bin/counterblock", line 9, in <module>
    load_entry_point('counterblock==1.1.0', 'console_scripts', 'counterblock')()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/__init__.py", line 5, in server_main
    server.main()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/server.py", line 128, in main
    StartUpProcessor.run_active_functions()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/processor/__init__.py", line 102, in run_active_functions
    func['function'](*args, **kwargs)
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/processor/startup.py", line 124, in start_api
    api.serve_api()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/processor/api.py", line 1655, in serve_api
    server.serve_forever()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/gevent/baseserver.py", line 284, in serve_forever
    self._stop_event.wait()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/gevent/event.py", line 77, in wait
    result = self.hub.switch()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/gevent/hub.py", line 331, in switch
    return greenlet.switch(self)
counterblock.lib.processor.startup.LinkedFailed: <Greenlet at 0x7f2934681410: process_cp_blockfeed(<zmq.green.core._Socket object at 0x7f29346cf870>)> failed with TypeError: not all arguments converted during string formatting

Also, please make sure you're running the latest commit from master, as I thought I fixed this last week (unless it is another similar issue)

HEAD of Federated Node is set to 074e5097ba7d883e7cd96ef78eee92434ac73f52, which is the latest.

robby-d commented 9 years ago

let me know if the above fix doesn't take care of it. thanks!

ArtemKo7v commented 9 years ago

Now it fails with another exception:

WARNING:blockfeed: counterpartyd MINOR DB version change (we built from 9.49, counterpartyd is at 9.50). Wiping our state data.
Traceback (most recent call last):
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/gevent/greenlet.py", line 327, in run
    result = self._run(*self.args, **self.kwargs)
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/blockfeed.py", line 205, in process_cp_blockfeed
    app_config = blow_away_db()
NameError: global name 'blow_away_db' is not defined
<Greenlet at 0x7f6541d9d410: process_cp_blockfeed(<zmq.green.core._Socket object at 0x7f6541de9940>)> failed with NameError

Traceback (most recent call last):
  File "/usr/local/bin/counterblock", line 9, in <module>
    load_entry_point('counterblock==1.1.0', 'console_scripts', 'counterblock')()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/__init__.py", line 5, in server_main
    server.main()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/server.py", line 128, in main
    StartUpProcessor.run_active_functions()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/processor/__init__.py", line 102, in run_active_functions
    func['function'](*args, **kwargs)
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/processor/startup.py", line 124, in start_api
    api.serve_api()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/processor/api.py", line 1655, in serve_api
    server.serve_forever()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/gevent/baseserver.py", line 284, in serve_forever
    self._stop_event.wait()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/gevent/event.py", line 77, in wait
    result = self.hub.switch()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/gevent/hub.py", line 331, in switch
    return greenlet.switch(self)
counterblock.lib.processor.startup.LinkedFailed: <Greenlet at 0x7f6541d9d410: process_cp_blockfeed(<zmq.green.core._Socket object at 0x7f6541de9940>)> failed with NameError: global name 'blow_away_db' is not defined
robby-d commented 9 years ago

ok, please reopen if there is another issue and I'll get that fixed as well. We've fixed our exception reporting code and these issues are now (helpfully) popping up due to the new code not silently suppressing exceptions

ArtemKo7v commented 9 years ago

Thanks a lot, seems to be working now!

ArtemKo7v commented 9 years ago

Another Federated Node, same issue, another exception:

INFO:startup: Starting up RPC API handler...
Traceback (most recent call last):
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/gevent/greenlet.py", line 327, in run
    result = self._run(*self.args, **self.kwargs)
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/blockfeed.py", line 163, in process_cp_blockfeed
    config.state['my_latest_block'] = database.rollback(my_latest_block['block_index'])
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/database.py", line 202, in rollback
    raise Exception("Can't roll back to specified block index: %i doesn't exist in database" % max_block_index)
Exception: Can't roll back to specified block index: 278270 doesn't exist in database
<Greenlet at 0x750bb5ff1410: process_cp_blockfeed(<zmq.green.core._Socket object at 0x750bb088aef0>)> failed with Exception

Traceback (most recent call last):
  File "/usr/local/bin/counterblock", line 9, in <module>
    load_entry_point('counterblock==1.1.0', 'console_scripts', 'counterblock')()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/__init__.py", line 5, in server_main
    server.main()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/server.py", line 128, in main
    StartUpProcessor.run_active_functions()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/processor/__init__.py", line 102, in run_active_functions
    func['function'](*args, **kwargs)
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/processor/startup.py", line 124, in start_api
    api.serve_api()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/counterblock/lib/processor/api.py", line 1655, in serve_api
    server.serve_forever()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/gevent/baseserver.py", line 284, in serve_forever
    self._stop_event.wait()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/gevent/event.py", line 77, in wait
    result = self.hub.switch()
  File "/home/xcp/federatednode_build/env.counterblock/local/lib/python2.7/site-packages/gevent/hub.py", line 331, in switch
    return greenlet.switch(self)
counterblock.lib.processor.startup.LinkedFailed: <Greenlet at 0x750bb5ff1410: process_cp_blockfeed(<zmq.green.core._Socket object at 0x750bb088aef0>)> failed with Excep                             tion: Can't roll back to specified block index: 278270 doesn't exist in database
robby-d commented 9 years ago

please try the additional fix in develop (if it works I'll promote it to master)

ArtemKo7v commented 9 years ago

It was helpful! Thank you!