MPOS mode needs to pool/flush mysql queries to prevent socket exhaustion

[miningpoolhub]

Hi I see some logs like "XXXXX has somehow had a socket error" Sometimes XXXXXX is user name, but sometimes it's just "undefined".

When I run a small pool, it's okay with just one process. When it gets bigger, the above message comes out frequently and I had to use cluster option to expand processes.

And, when it gets much bigger, I had to create x4 processes. I am using 8 core server, so I create 32 node child processes.

The problem is, it continually happen time to time even running 32 node processes. My node-stratum based coin workers are about 300 concurrently, so it's not that big.

I think the problem is, client waits new job but node process not responds in some amount of time. (I adapted latest "30 seconds resend job" code also but this problem not solved)

The main problem might be the DB works since it inserts many shares for MPOS compatible in synchronous. It may block thread for a while, isn't it?

Does this happen to just normal node-stratum only? Or only for MPOS compatible stratum servers.

I want to know problems in detail. Thanks.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[zone117x]

I just did a commit https://github.com/zone117x/node-stratum/commit/98d3dcc5325f43dfa205697be4498bd17fea44b6 which added more verbose error logging so you can see exactly what type of socket error happens now. Please report back the full error messages you should be seeing now

[miningpoolhub]

OK I will test it. Thanks.

[miningpoolhub]

I got this message.

[aphroditecoin (Fork 5)] username.3 has somehow had a socket error: {"code":"ETIMEDOUT","errno":"ETIMEDOUT","syscall":"read"}

I replaced real username to just "username", but it doesn't matter I think.

[zone117x]

Looks like that error message is simply just the miner/client disconnecting/timing out in an odd way. I don't think its a problem with the stratum server. I should probably handle that error better and just emit the standard "client disconnected" message when that happens.

[miningpoolhub]

I don't think all of this kind logs are normal behavior since some user reported that their miners are disconnected and connects to failover server. As I expand the child processes this problem occurred much lower and got feedback from miners that things are going well.

It maybe not the stratum server problem if this was not reported until now but it might be the problem of MPOS compatibility layer things I think. Please don't change the log message before this issue is well explained or solved. We would think that there's no problem if the logs are just saying "disconnected".

[zone117x]

The MPOS mode is not optimized at all. Most importantly, the mysql share inserts need to be pooled and flushed to db on some configured interval so that every share isn't creating a mysql request. Perhaps someone else will be kind enough to implement that and do a pull request - otherwise I will only be able to do it when I have the time.. which won't be for a while.

[miningpoolhub]

Thanks anyway. This project is awesome. I will contribute someday not now since I'm not familiar with nodejs. I'll learn :)

[feeleep75]

hi - just wanted to bump it and ask if anyone looked at improving mpos integration already

[zone117x]

Hey I'm not interested in working on MPOS mode issues unless a bounty is placed. I created one here: https://www.bountysource.com/issues/1687965-mpos-mode-needs-to-pool-flush-mysql-queries-to-prevent-socket-exhaustion

If enough people contribute to reach the bounty limit then I will implement this feature. I suggest also posting this issue to the NOMP repo https://github.com/zone117x/node-open-mining-portal