Memory depletion during the synchronization of the Shiden network node

[TomaszWaszczyk]

Description

I try to make a fully sync of Shiden network, but I am not able to do it.

Steps to Reproduce

All made according to documentation with and without wrap mode with and without -- --wrap flag.

Expected vs. Actual Behavior

Obtain fully sync node.

Environment

Describe the environment in which you encountered this bug. Use the list below as a starting point and add additional information if you think it's relevant.

• Operating system: GNU Debian 12.2
• Project version/tag: v5.28.0 but I was trying also with a few last versions, too.

Additional Information

Initially, I attempted to synchronize with 32GB of RAM and 32GB of swap, contemplating whether expanding the swap to 64GB could alleviate the issue. Regrettably, even with an additional 64GB of swap, the problem persisted. Out of memory occurs around after ~40hrs of synchronization.

[ashutoshvarma]

Can you share the logs and the full command used to start the node?

[TomaszWaszczyk]

Commands like:

./astar-collator \
  --collator \
  --name 'test-name' \
  --chain shiden \
  --base-path /var/lib/astar \
  --pruning archive \
  --trie-cache-size 0 \
  --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0'

or

./astar-collator \
  --collator \
  --name 'test-name' \
  --chain shiden \
  --base-path /var/lib/astar \
  --trie-cache-size 0 \
  --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0' \
  -- \
  --warp

or

ExecStart=/home/collator/astar-collator \
            --chain 'shiden' \
            --rpc-cors all \
                        --base-path '/home/shiden/' \
                        --name 'name' \
                        --rpc-port 9933 \
            --state-pruning archive \
            --blocks-pruning archive \
            --prometheus-external \
                        --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0' \
            -- --sync warp

Unfortunately, I don't have logs as I've removed them. I try sync next time, and if I have logs, I'll update the issue.

[asggWa]

Voy a intentar reproducirlo. Es shibuya es evm. Supongo que no habrás confundido la main nework con la testnetwork en el provider metamask o cuál séa. De modo que al haber tenido que formatear tenía que deciddirme. Quzá tarde yo en construir que vosotros en resolver, pero si lo consigo ya tendr'el entorno de desarrollo que quiero; uno de ellos.

[asggWa]

Supongo que el fallo es que tengo astar nativo básico instalado y me falta seguir el manual de configuración del astar-collator. Ya lo tuve en el pasado. Volveré a poder tenerlo si no explota mi pc. Mientras tanto podéis investigar mis intentos fallidos.

treebtc@treebtc:~$ sudo ./polkadot \
  --collator \
  --name 'test-name' \
  --chain shiden \
  --base-path /var/lib/astar \
  --pruning archive \
  --trie-cache-size 0 \
  --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0'
[sudo] contraseña para treebtc: 
./polkadot: 1: Not: not found
treebtc@treebtc:~$ su
Contraseña: 
root@treebtc:/home/treebtc# ./polkadot \
  --collator \
  --name 'test-name' \
  --chain shiden \
  --base-path /var/lib/astar \
  --pruning archive \
  --trie-cache-size 0 \
  --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0'
./polkadot: línea 1: Not: orden no encontrada
root@treebtc:/home/treebtc# polkadot   --collator   --name 'test-name'   --chain shiden   --base-path /var/lib/astar   --pruning archive   --trie-cache-size 0   --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0'
error: unexpected argument '--collator' found

Usage: polkadot [OPTIONS]
       polkadot <COMMAND>

For more information, try '--help'.
root@treebtc:/home/treebtc# ./polkadot \
  --collator \
  --name 'test-name' \
  --chain shiden \
  --base-path /var/lib/astar \
  --trie-cache-size 0 \
  --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0' \
  -- \
  --warp
./polkadot: línea 1: Not: orden no encontrada
root@treebtc:/home/treebtc# polkadot   --collator   --name 'test-name'   --chain shiden   --base-path /var/lib/astar   --trie-cache-size 0   --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0'   --   --warp
error: unexpected argument '--collator' found

Usage: polkadot [OPTIONS]
       polkadot <COMMAND>

For more information, try '--help'.
root@treebtc:/home/treebtc# polkadot   --name 'treebtc'   --chain shiden   --base-path /var/lib/astar   --trie-cache-size 0   --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0'   --   --warp
error: unexpected argument '--warp' found

Usage: polkadot [OPTIONS]
       polkadot <COMMAND>

For more information, try '--help'. root@treebtc:/home/treebtc# polkadot --name 'treebtc' --chain shiden --base-path /var/lib/astar --trie-cache-size 0 --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0' --
Error: 0: Invalid input: Error opening spec file shiden: No such file or directory (os error 2)

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it. Run with RUST_BACKTRACE=full to include source snippets. root@treebtc:/home/treebtc#

Yo supongo que algun día aprenderé a poner los 3 ''' como lo hace el "<>" sin necesidad de usar copy paste. Mientras tanto pues de mi lado el compilador sergirá cascando. 

[asggWa]

curioso numero #1117. Es mayor que 1024 que era el tamaño máximo de página. tabla de mapa de pagina o mapa de tabla del paginación hasta que fué aumentado a 4Gib y ahora no sé ni por dónde irá. Sé que uso un ssd de 320 gb para emular otros 32 gb de ram que es la ram física que tengo. Desconozco si resuelve nada, lo que sí es seguro que almacena ram a un almacenamiento ra+ access.

[asggWa]

ah, por cierto. ahora que recuerdo. recién acabo de formatear. consigo rodar polkadot desde usr/bin/polkadot, creo que faltan los ln o link simbolicos para que tire desde dńde he tratado de arrancar anteriormente. Seguiré trabajando. Ahora... No tengo ni para pan ni para dentista y necesito ambos o dinero como el comer. Pasaría otra vez la wallet, pero me duele pedir por la cara com el comer. Literalmente.Aunque sanchinflas and company me obliguen a pasar dolor porque a ellos les da gustito pensarse superiores hasta despues que dios los pase por la quilla al infierno. Acerca de mi ... El mañana no es suficiente. El mañana nunca muere. y will reborn, sayonara capuyo. You' ll never walk alone.

[asggWa]

de aquí chutaba, pero algo me dice que maás valía pararlo porque es verdad que al al tema de los certificados no he llegado en esta era. aquí la salida rarita. yarn run v1.22.19 warning package.json: No license field error Command "build" not found. info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. root@treebtc:/home/treebtc# polkdadot polkdadot: orden no encontrada root@treebtc:/home/treebtc# ./polkdadot --name treebtc bash: ./polkdadot: No existe el archivo o el directorio root@treebtc:/home/treebtc# cd /usr/bin root@treebtc:/usr/bin# ./polkadot --name treebtc 2024-01-18 08:25:18 Parity Polkadot 2024-01-18 08:25:18 ✌️ version 1.5.0-a3dc2f15f23 2024-01-18 08:25:18 ❤️ by Parity Technologies <admin@parity.io>, 2017-2024 2024-01-18 08:25:18 📋 Chain specification: Polkadot 2024-01-18 08:25:18 🏷 Node name: treebtc 2024-01-18 08:25:18 👤 Role: FULL 2024-01-18 08:25:18 💾 Database: RocksDb at /root/.local/share/polkadot/chains/polkadot/db/full 2024-01-18 08:25:26 🔨 Initializing Genesis block/state (state: 0x29d0…4e17, header-hash: 0x91b1…90c3) 2024-01-18 08:25:26 👴 Loading GRANDPA authority set from genesis on what appears to be first startup. 2024-01-18 08:25:27 👶 Creating empty BABE epoch changes on what appears to be first startup. 2024-01-18 08:25:27 🏷 Local node identity is: 12D3KooWEBSZL12B7UxhEXcgEMK1cR26qkoafvN3s7yrEozWRp5b 2024-01-18 08:25:28 💻 Operating system: linux 2024-01-18 08:25:28 💻 CPU architecture: x86_64 2024-01-18 08:25:28 💻 Target environment: gnu 2024-01-18 08:25:28 💻 CPU: Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz 2024-01-18 08:25:28 💻 CPU cores: 4 2024-01-18 08:25:28 💻 Memory: 32000MB 2024-01-18 08:25:28 💻 Kernel: 6.5.0-14-generic 2024-01-18 08:25:28 💻 Linux distribution: Ubuntu 22.04.3 LTS 2024-01-18 08:25:28 💻 Virtual machine: no 2024-01-18 08:25:28 📦 Highest known block at #0 2024-01-18 08:25:28 〽️ Prometheus exporter started at 127.0.0.1:9615 2024-01-18 08:25:28 Running JSON-RPC server: addr=127.0.0.1:9944, allowed origins=["http://localhost:*", "http://127.0.0.1:*", "https://localhost:*", "https://127.0.0.1:*", "https://polkadot.js.org"] 2024-01-18 08:25:28 🏁 CPU score: 343.40 MiBs 2024-01-18 08:25:28 🏁 Memory score: 7.78 GiBs 2024-01-18 08:25:28 🏁 Disk score (seq. writes): 93.79 MiBs 2024-01-18 08:25:28 🏁 Disk score (rand. writes): 41.41 MiBs 2024-01-18 08:25:28 🥩 BEEFY gadget waiting for BEEFY pallet to become available... 2024-01-18 08:25:29 Sending fatal alert BadCertificate 2024-01-18 08:25:29 🔍 Discovered new external address for our node: /ip4/84.122.98.107/tcp/30333/ws/p2p/12D3KooWEBSZL12B7UxhEXcgEMK1cR26qkoafvN3s7yrEozWRp5b 2024-01-18 08:25:33 ⚙️ Syncing, target=#19092289 (6 peers), best: #2280 (0x6102…214f), finalized #2048 (0x4d5c…c532), ⬇ 351.7kiB/s ⬆ 33.0kiB/s 2024-01-18 08:25:38 ⚙️ Syncing 563.0 bps, target=#19092290 (7 peers), best: #5095 (0x1417…d4c9), finalized #4608 (0x1079…df89), ⬇ 195.1kiB/s ⬆ 1.8kiB/s 2024-01-18 08:25:43 ⚙️ Syncing 562.8 bps, target=#19092291 (7 peers), best: #7909 (0x3ef8…10c2), finalized #7680 (0x028f…5be1), ⬇ 244.6kiB/s ⬆ 28.7kiB/s 2024-01-18 08:25:48 ⚙️ Syncing 568.6 bps, target=#19092292 (7 peers), best: #10752 (0x9e6a…a91f), finalized #10240 (0x13ee…c660), ⬇ 212.2kiB/s ⬆ 4.3kiB/s 2024-01-18 08:25:53 ⚙️ Syncing 530.8 bps, target=#19092292 (8 peers), best: #13406 (0x0e14…f30e), finalized #13312 (0x9b29…14d4), ⬇ 179.9kiB/s ⬆ 2.3kiB/s 2024-01-18 08:25:58 ⚙️ Syncing 582.2 bps, target=#19092293 (9 peers), best: #16317 (0xc0d4…52f9), finalized #15872 (0x4305…8b01), ⬇ 208.8kiB/s ⬆ 2.7kiB/s 2024-01-18 08:26:03 ⚙️ Syncing 574.8 bps, target=#19092294 (9 peers), best: #19191 (0xcd05…e9dc), finalized #18944 (0x48b9…c40b), ⬇ 204.4kiB/s ⬆ 3.1kiB/s 2024-01-18 08:26:08 ⚙️ Syncing 579.0 bps, target=#19092295 (9 peers), best: #22086 (0x4d38…fdce), finalized #22016 (0x91e0…d979), ⬇ 203.1kiB/s ⬆ 1.5kiB/s ^Z [1]+ Detenido ./polkadot --name treebtc root@treebtc:/usr/bin#

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 7 days with no activity.