openethereum / parity-ethereum

The fast, light, and robust client for Ethereum-like networks.
Other
6.81k stars 1.68k forks source link

Out Of Memory #11815

Closed fourseaLee closed 4 years ago

fourseaLee commented 4 years ago

Before filing a new issue, please provide the following information.

If you think that your issue is an exploitable security vulnerability, please mail your bugreport to security@parity.io instead; your submission might be eligible for our Bug Bounty. You can find mode info on the reporting process in SECURITY.md

Your issue description goes here below. Try to include actual vs. expected behavior and steps to reproduce the issue.

I run a node with params with "--base-path=../paritydata/" . Then I have two daemon to get balance and scan block from rpc. My machine has 16G memory , 8 core cpu and 460G nvme ssd. After a night ,the node is down ,Maybe the reason is "Out Of Memory". Because I found when I call rpc , the memory is only increasing,not decreasing.

adria0 commented 4 years ago

@fourseaLee, we experienced some problems with 16GB of ram, and snapshot creation. Could you try running it with --no-periodic-snapshotflag activated, please? Your feedback about this change will be very appreciated.

fourseaLee commented 4 years ago

I don't think snapshot results in server down. Because I have runned it on local server (ubuntu 18 server), but not do rpc, which has been running two weeks not down. When I do rpc interface(eth_getbalance and eth_getBlockByNumber) incessantly,The node will be down about every 8 hour. I try to use your params (--no-periodic-snapshot) again,Maybe good.

adria0 commented 4 years ago

So, if I understand correctly, the memory is stable if you do not run the eth_getbalance and eth_getBlockByNumber isn't it?

fourseaLee commented 4 years ago

yeah, you are right. I have used your params (--no-periodic-snapshot) again, but still have the problem(out of Memory). but local node is running ok of three week ,(not do rpc [eth_getbalance eth_getBlockByNumber ] ).

vietthang207 commented 4 years ago

image Same thing happenned to me, this is server's metric recorded by Prometheus. All of my node is running 3.0.1. The RAM usage did not increase during the syncing phase, it only went up when I started to route RPC traffic to it.

vietthang207 commented 4 years ago

I have 2 nodes running 3.0.1, both go down due to out of memory. After restarting, one node recovers, the other stuck at syncing an old block and only escape that syncing loop after 22-23 hours.

hlozadaccs commented 4 years ago

@adria0 Same problem around here, what are the consequences of using the flag --no-periodic-snapshot? after applying the changes, memory stabilized at 3.06 GB

vietthang207 commented 4 years ago

@adria0 Same problem around here, what are the consequences of using the flag --no-periodic-snapshot? after applying the changes, memory stabilized at 3.06 GB

Thank you! --no-periodic-snapshot also do the trick for me. No more out of memory. So this snapshot seems to be the problem?

adria0 commented 4 years ago

@imatsu , @vietthang207, yes, it has been creating problems to a lot of users. This is mainly interesting for boot nodes at this moment.

rakita commented 4 years ago

Seems connected to #11760