search

dashpay / dash

Dash - Reinventing Cryptocurrency
https://www.dash.org
MIT License
1.49k stars 1.2k forks source link

CPU/Memory Problem after two days, process shutdown #535

Closed minersday closed 9 years ago

minersday commented 9 years ago

The CPU hogging dashd thread is, please fix it, if possible: from top -H: 21921 xxxxx 22 2 908200 179908 36092 R 58.1 8.8 49:39.70 dash-msghand

getting messages like this, continious flood, like screen is scrolling as fast as it can.. strace -p 21921: endto(39, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(31, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(35, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(36, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(37, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(33, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(20, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(26, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(24, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(28, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(30, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(32, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(21, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(23, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(29, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(27, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(39, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(31, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(35, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(36, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(37, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(33, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\356\247\370\"\16\274\242\260Q\r\220\336"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(20, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(26, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(24, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(28, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(30, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(32, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(21, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(23, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(29, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(27, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(39, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140 sendto(31, "\277\fk\275dseep\0\0\0\0\0\0\0t\0\0\0\373\257\264\272,\226\312<\375\35\242\304"..., 140, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 140

minimal hogging: futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140475, {1439662667, 560970182}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140477, {1439662667, 562308164}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140479, {1439662667, 563645382}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140481, {1439662667, 565038070}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140483, {1439662667, 566340425}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140485, {1439662667, 567818400}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140487, {1439662667, 569213288}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140489, {1439662667, 570554175}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140491, {1439662667, 571893022}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140493, {1439662667, 573248748}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140495, {1439662667, 574590053}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0 futex(0x7f615b513d94, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 34140497, {1439662667, 575932126}, ffffffff) = -1 ETIMEDOUT (Connection timed out) futex(0x7f615b513d68, FUTEX_WAKE_PRIVATE, 1) = 0

Don't know if it's from BOOST_FOREACH doing the dseep looping, maybe wait/sleep has to be in there?

minersday commented 9 years ago

Issue fixed with version ..45

darkcoinproject commented 9 years ago

Perfect thanks

On Sun, Aug 16, 2015 at 1:33 PM, minersday notifications@github.com wrote:

Issue fixed with version ..45

— Reply to this email directly or view it on GitHub https://github.com/dashpay/dash/issues/535#issuecomment-131618431.

minersday commented 9 years ago

There's something happening after two day's in 0.45 release, it seems CPU / Memory goes up and down like a chainsaw..

spamming the same thing connection timed out.

minersday commented 9 years ago

cpu-usage memory-usage

minersday commented 9 years ago

Spamming the (Connection timed out), in the dash-msghand process thread. And memory usage seems to be going up.

eduffield222 commented 9 years ago

Do you have a "connect=" in your configuration?

minersday commented 9 years ago

Nope, in any of the dash.conf:s.

minersday commented 9 years ago

People having daemon shutdown, could be because of the memory/CPU erratic use after two days. Could trigger daemon shutdown in hosting services. I don't have that problem, hosting my own instances..

minersday commented 9 years ago

Could this be the problem? http://stackoverflow.com/questions/28320969/futex-return-etimedout-quicker-than-the-parameter-timeout

all of masternodes are configured to keep the time with ntpd, and in VM-hosting enviroments, there are clock cycle skips, because of the shared enviroment. And ntpd needs to adjust time.

minersday commented 9 years ago

Changed the clocksource in VM to acpi_pm: echo acpi_pm > /sys/devices/system/clocksource/clocksource0/current_clocksource

now getting mostly spammed in strace with: gettimeofday({1439921834, 663640}, NULL) = 0 gettimeofday({1439921834, 663765}, NULL) = 0 gettimeofday({1439921834, 663841}, NULL) = 0

minersday commented 9 years ago

With .046 CPU/MEM/Disk/Network usage, seems to gone back down to normal levels and not going up and down rapidly.