Open roderik opened 7 years ago
Do you have some Python tool that lets you trace which piece of code it's stuck on?
I can install and configure anything we need to get to the bottom of this, but I have no clue on the python ecosystem.
An educated guess, all these transactions are in one stream. Abe can handle the bitcoin blockchain with 90gb even on sqlite. So probably in the additions for streams that are triggered for the homepage.
You should be able to use the basic pdb Python debugger – see the documentation here:
https://docs.python.org/2/library/pdb.html
We haven't tried it yet, but you should be able to add an import pdb; pdb.set_trace()
at the top of the main Mce/abe.py
file, then re-run the installation instructions in the Explorer README, then use the continue
command in the debugger to let the Explorer start running, then when it looks stuck, use the where
or list
commands to see which piece of code it's stuck on.
@roderik Are you using dummy/test data? If yes, maybe you could share all the chain and explorer data so we can try and replicate.
Have a similar issue.
Debugged a little:
2019-09-20 10:11:31.953006
getting num_txs
2019-09-20 10:11:31.985604
getting num_addresses
2019-09-20 10:16:05.755095 <- took almost 5 mins
getting num_peers
2019-09-20 10:16:05.821592
getting num_assets
2019-09-20 10:16:05.876624
getting num_streams
2019-09-20 10:16:21.742359
got all nums
2019-09-20 10:16:21.743385
getting mempool
2019-09-20 10:16:21.791694
getting recenttx
2019-09-20 10:17:47.790923 <- took more than 1 min
getting sorted_mempool
127.0.0.1 - - [20/Sep/2019 10:17:47] "GET / HTTP/1.1" 200 6175
These queries look suboptimal when you have a significant amount of transactions (as any DISTINCT
query on a complex table/view without proper indexes would be):
In my case it's actually not that large but already quite problematic:
sqlite> select count(*) from txout_detail;
1758971
Did anyone have time to look at this issue? It looks critical for networks with significant number of transactions. @gidgreen @bitcartel
This should be fixable by adding the appropriate indexes at the time of table creation. Do you want to try to add these indexes during initialization in DataStore.py
, restarting the Explorer afresh, and confirming?
Adding indexes didn't help. Implemented a workaround by using more optimal queries: https://github.com/chainstack/multichain-explorer/pull/4.
I'm running a quite active network (20tx/block, 10s blocktime, 124k blocks) and the explorer has died completely.
The log is full of the following, but no more pages are being served.
curl -i http://localhost:2750
just stallsThe explorer just goes to 100% CPU on one core and stays there indefinitely.
Any clue on how to revive the explorer?