tfreedman / explorer

An open source cryptocurrency block explorer
MIT License
8 stars 18 forks source link

explorer under heavy load #9

Open xaya opened 5 years ago

xaya commented 5 years ago

we are using a fork of this explorer at explorer.xaya.io (with a namecoin fork chain)

If someone tries to check a mining address or similar (that has thousands of txs) it causes this, and i suspect someone can do it intentionally.

This website is under heavy load (queue full) We're sorry, too many people are accessing this website at the same time. We're working on this problem. Please try again later.

any ideas? we are willing to donate

i see our daemon (xayad) using a lot of CPU when it happens and also Ruby.

tfreedman commented 5 years ago

This is a bug that I'm aware of, but there's no easy fix. It's caused by long-running queries that keep running even when the client that requested them is long gone. For some reason they aren't getting dropped from the queue, which makes nginx / passenger think that process is still busy, clogging the request queue.

It might be possible to change the nginx / Passenger config to consider a request dead after a certain amount of time, though I don't know the exact config option. Another approach is to trigger a timeout in the app (which would return an error page after N seconds if the request isn't done processing, which at least saves the server even though it's worthless to the user).

The best approach would be to prevent long running queries from happening in the first place, by paginating queries for things like unknown addresses (which could have thousands of transactions), via SQL LIMIT(). Unfortunately, this is trickier than it sounds because it means you have to compute the running debit/credit address total to calculate the balance on the page, instead of just summing the credit and debit columns after retrieving all results. I tried to implement this as a fix at one point, but ran out of time to finish it. Unfortunately, I left the company that sponsored me to develop this explorer, and don't foresee having time to implement this fix anytime soon.