search

spesmilo / electrumx

Alternative implementation of spesmilo/electrum-server
MIT License
436 stars 349 forks source link

Unresponsive as new block is processed? #111

Closed FMCorz closed 3 years ago

FMCorz commented 3 years ago

Hi,

I'm running into issues where RPC clients connected to ElectrumX are suddenly unable to get a response. The server does not respond and the client times out its request. Generally, a log in ElectrumX will also show that the request timed out, although the client gives up before that log appears.

What seems to happen is that the same request will fail shortly after ElectrumX announces that it has detected a new block. My assumption is that the processing of the block creates a lock that blocks the server from responding to RPC requests. I am not sure what best ways to reproduce this are, but to confirm that it wasn't my client implementation that was faulty, I did a getaddresshistory() in Electrum's console as my clients were timed out, and Electrum froze for along period of time and never got a response.

I don't know whether this behaviour is intentional, or if it is a side effect of misuse or misconfiguration on my end, so I would appreciate some insights. However, if the blocking is real and clients cannot interact with the server for as long as new blocks are computed, then I would classify this as a significant issue as, in my case, the server is blocked for 1-2 mins, and with blocks every 10 mins that is a lot of down time.

In the best of worlds, ElectrumX would switch to a read-only state during which it is capable of accessing its own database. The alternative would be to run another ElectrumX as a slave that does not process new blocks, and becomes available while the master is busy.

I'm more than happy to offer my help.

Thanks!

SomberNight commented 3 years ago

I am not sure why the server might stop serving client requests while it is processing a new block. I don't think there is code that intentionally does this.

I would say bitcoind on the other hand might block serving RPC requests while processing a block, at least certainly RPCs that take the main lock. However getaddresshistory() should not hit bitcoind at all.

Do you experience this on a public server with many clients connected; or can you also reproduce in an isolated environment where you control all clients?

in my case, the server is blocked for 1-2 mins

It should only take a few seconds to process a block. What kind of hardware do you have?

FMCorz commented 3 years ago

First of all, let me thank you for responding so quickly!

To cut the story short, the problem was on my end. Not in the client, not in ElectrumX, but instead in another service running on the same hardware that was intermitently using 100% of CPU. It was another ElectrumX instance running for another blockchain and it needed to be compacted so it constantly shutdown and restarted, and that would have appeared to be related to a new block.

My apologies for taking up some of your time, but your answers confirming that there were not thread lock lead to investigate in different areas and finding the problem!

Thanks again!