Memory Leak when doing normal RPC calls to node

[journeytosilius]

System information

Erigon version: ./erigon --version

erigon version 2.54.0-aeec5221

OS & Version: Windows/Linux/OSX

Linux

Commit hash:

aeec5221

Erigon Command (with flags/config):

./erigon/build/bin/erigon --datadir mainnet_data --http --http.addr 192.168.1.69 --http.port 8545 --ws --torrent.download.rate=60mb --authrpc.jwtsecret=/home/funk/crucialp5/prysm/jwt.hex --authrpc.addr=0.0.0.0 --authrpc.vhosts=*

Consensus Layer:

prysm

Consensus Layer Command (with flags/config):

./prysm.sh beacon-chain --execution-endpoint=http://192.168.1.69:8551 --suggested-fee-recipient=0xEa6D2a732700B387BCD259AE2d25805E3fa1b9BE --checkpoint-sync-url=https://beaconstate.ethstaker.cc/ -genesis-beacon-api-url=https://beaconstate.ethstaker.cc/ --jwt-secret=jwt.hex

Chain/Network:

Ethereum

Expected behaviour

I have a client API that I developed in Geth and worked just fine. Decided to move to Erigon for archive node. When it startup, it collects data from factories and instantiates pair objects, and does things like update reserves, in goroutines. The expected behaviour is that it runs just like it did with Geth, where I collect data in parallel and then I do statistic operations with it. Since Erigon doesn't seem to support IPC sockets, I have configured the connection to the node via websockets.

Actual behaviour

Erigon starts taking more and more RAM until the server starts swapping and gets rendered unusable. Since it works with a full Geth node, I wonder what would be the cause of this, and if maybe it's been fixed in some devel branch commit.

Steps to reproduce the behaviour

Maybe just try to get the reserves of 3K pairs or more in a go-routine, or something similar ( heavy parallel calls ) And also, subscribe to events of 3-5K pairs or more

Backtrace

[backtrace]

[AskAlexSharov]

IPC - does support - see last release notes

Can see where leaking ram by: add --pprof flag run go tool pprof -inuse_space -png http://127.0.0.1:6060/debug/pprof/heap > mem.png and share picture.

[journeytosilius]

when I took this, it has taken like 40 GB RAM and kept on going non-stop. I can only stop the crash If I shutdown Erigon

[journeytosilius]

I saw in some other issue that there are problems with eth_newFilter, is it possible that it comes from there ? Is this bug already identified ? If so, is there any patch for it ? I'm using autogenerated bindings for the event subscription. Can I circumvent this by avoiding the autogen bindings maybe ?

[journeytosilius]

update : this only happens with event receivers. I have receivers for Mint, Burn Sync, Swap, Transfer. To test further, I have first tested without receivers which yields zero problems. After that I have activated only Syncs. The same happens, it takes way too much RAM at initialization, but then, interestingly it gets almost completely garbage collected and it goes back to normal. So the problem seems to be at the initialization only, but gets aggravated by having more types of receivers. So, it does some crazy allocation ( which doesn't happen in geth ) and then it gets garbage collected. This is still a problem, because you might need 1TB of RAM just to initialize 100K receivers or so

[AskAlexSharov]

Let’s do: go tool pprof -inuse_space -png -lines -focus=SubscribeLogs http://127.0.0.1:6060/debug/pprof/heap > mem.png

[journeytosilius]