search

zone117x / node-cryptonote-pool

Mining pool for CryptoNote based coins such as Bytecoin and Monero
GNU General Public License v2.0
826 stars 1.14k forks source link

API becomes very slow on 1000+ connected miners #238

Open easyhash opened 6 years ago

easyhash commented 6 years ago

We experience that the pool API is very slow when many miners are connected. We currently have 3000-3500 miners connected and our API use up to 8 seconds to respond.

Pool is running in a Docker environment, connecting to a Docker Redis4. Does not look like a hardware issue. Pool is on a dedicated server and only using ~20-40% CPU and a fraction of RAM.

API URL: https://api-etn.easyhash.io/stats

From inside the Docker container:

root@d6dab5e94cec:/pool# time curl http://localhost:8443/stats 
...
real    0m3.430s
user    0m0.004s
sys     0m0.000s

Any pointer on where the bottleneck is?

xstalpha commented 6 years ago

Bandwidth? Hard drive speed?

lukaszmatczak commented 6 years ago

I've discovered that the problem is here: https://github.com/zone117x/node-cryptonote-pool/blob/69ef5768aeb92966b527f8836b003e163d7a222e/lib/api.js#L48

With large amount of miners, redis needs more time to return results and whole API is being blocked by this period of time.

phenomax commented 6 years ago

I tried to improve the performance of redis through my PR. What about your local hosted redis instance... does it have enough memory and cpu? Please provide insight information about your instance by executing the INFO command in the redis-cli.