search

opentensor / subtensor

Bittensor Blockchain Layer
The Unlicense
131 stars 147 forks source link

node-subtensor memory leak when running lite-node on mainchain #696

Open baonq-me opened 1 month ago

baonq-me commented 1 month ago

Describe the bug

Memory usage of node-subtensor is rising continuously.

To Reproduce

Start a lite node on the mainchain by following this instruction https://docs.bittensor.com/subtensor-nodes/using-source

commit 1332d077ea73bc7bf40f551c7f1adea3370df2bd (HEAD -> main, origin/main, origin/HEAD)
Merge: aec052a0 99253056
Author: Unconst <32490803+unconst@users.noreply.github.com>
Date:   Tue Jul 23 16:32:49 2024 -0500

    Merge pull request #657 from opentensor/rajkaramchedu-patch-1-1

    Update README.md

Expected behavior

Memory usage should keep around several GBs

Screenshots

CleanShot 2024-07-30 at 10 10 11@2x

CleanShot 2024-07-30 at 10 17 53@2x

Environment

Ubuntu 24.04, arm64 architecture, rustc 1.80.0

Additional context

CleanShot 2024-07-30 at 10 13 13@2x

subtensor-exporter.txt

SFuller4 commented 1 month ago

Could this be causing the panics that people have been experiencing as tagged in #655? If the memory is still being held onto after a restart it would explain the continuing issues.

orriin commented 1 month ago

Could be related. Thanks @baonq-me for reporting, will investigate this.

baonq-me commented 1 month ago

34GB more since yesterday

CleanShot 2024-07-31 at 09 36 49@2x

distributedstatemachine commented 1 week ago

@baonq-me is this still an issue ?