Snipa22 / nodejs-pool

Other
480 stars 423 forks source link

Worker module crashed #432

Open LegolasGChief opened 6 years ago

LegolasGChief commented 6 years ago

The worker failed to return last block header. Please verify if the daemon is running properly.

Anyone have any ideas why? how often should I restart modules have been running for about a week and recently had pretty big jump in hashrate, CPU usage htop is minimal yet modules crashing? :(

I check daemon it is still at right block height

snoski3 commented 6 years ago

@LegolasGChief Were you able to upgrade to 12.2? I noticed I had the same issue.

shigutso commented 6 years ago

I'm also having this issue, but my CPU is being used a lot. (Aeon pool)

Worker module is crashing, like it's losing connection with the daemon, but the daemon is running fine, without errors.

snoski3 commented 6 years ago

I ran a git pull, which works. Then I ran npm install, which doesn't work. $ npm install audited 1067 packages in 2.497s found 23 vulnerabilities (9 low, 13 moderate, 1 high) run npm audit fix to fix them, or npm audit for details

I'm not sure if I need to start from scratch or update the modules manually.

snoski3 commented 6 years ago

I think if we can replace the parserjson module, we should be able to upgrade. High │ Regular Expression Denial of Service │ ├───────────────┼──────────────────────────────────────────────────────────────┤ │ Package │ parsejson │ ├───────────────┼──────────────────────────────────────────────────────────────┤ │ Patched in │ No patch available │ ├───────────────┼──────────────────────────────────────────────────────────────┤ │ Dependency of │ socketio

Also, I thought I upgraded to 12.0 but I checked the version in the Monero folder in the main folder and it is still version 11.

snoski3 commented 6 years ago

@LegolasGChief I upgraded NPM to version 6.1. If I didn't, I'm sure I wouldn't have these issues.

@Snipa22 Will I have to start from scratch because I upgraded NPM to version 6.1 or are there plans to patch the vulns in the dependencies?

Thanks,

LegolasGChief commented 6 years ago

This was for Aeon pool. Daemon is also using a lot if CPU sometimes not sure why maybe it is related to that

bobbieltd commented 6 years ago

Worker crash !?! Undefined of property blockID or something like that !?! It is not pool’s problem ( all those NPM stuffs are unrelated and OK ). Lost connectons with daemon happens some times ( yeah it can crash worker ) but it is OK. Daemon oftens have CPU usage surges. Nothing to worry.

bobbieltd commented 6 years ago

By the way, if someone here are using MoneroOcean’s repo, I can pull request the fix to avoid worker crashes 😌

snoski3 commented 6 years ago

@bobbieltd Has anyone successfully updated to 12.1? I went through the process back in April. I just found out the the daemon version is still 11.1

shigutso commented 6 years ago

moneromooo gave a workaround for this (it works for me):

go to the running daemon (aeond or monerod) and run the command print_pool_stats, if there are a lot of tx(es), it's probably what is causing the high usage of CPU and crashing the pool modules. To workaround this, run this command on the daemon: flush_txpool. As soon as I do this, the CPU usage of my server goes down to 10% and the modules stop crashing.

bobbieltd commented 6 years ago

Sometimes, transactions got stuck at txpool but it is rare. flush_txpool (you need to manually resend the stuck transactions to miners ) or try to restart coin daemon. Anyway, it can not cause worker died ( except Out of memory )

shigutso commented 6 years ago

The worker module crashes because of lack of CPU. I realized that if your server has more than 2 cores the modules don't crash. So to stop the CPU spike, flush_txpool works for me, because my server has only 2 cores.

DurinsMine commented 6 years ago

Seems like this is something to do with daemon rather than the pool lets hope devs are able to fix it

bobbieltd commented 6 years ago

If any module crashes, there is pm2 log error and you can know why.

DurinsMine commented 6 years ago

@bobbieltd the daemon is checking tx inputs and becoming unresponsive using too much cpu. The worker module fails to getBlockHeader which then causes the blockmanager module to crash. It says that there is no active block template or header if restart modules while its going crazy. Need to flush_txpool and then pm2 restart all.

There is a related but slightly different problem with pool software. The two slew fee options generating the larger input txs because of fractional fees being taken for the block payments to miners which then causes the daemon/pool to stall/crash so its kind of the root cause of above problem, a knock on effect I guess.

This is just info I have gotten from camping in IRC