unable to obtain and/or process more than 40 new blocks per day

nkinnan commented 2 years ago

This is on a pi 4 with 4 gb ram running https://github.com/monero-ecosystem/PiNode-XMR with usb ssd. The monerod build is obtained via "git clone --recursive -b $RELEASE https://github.com/monero-project/monero.git" according to the scripts. Stats dump after 20 hours from the web portal:

  Monero Version: 0.17.3.0-release
  Node Status: OK
  Busy Syncing: false
  Current Sync Height: 2537680
  P2P Outgoing Connections: 2
  P2P Incoming Connections: 7
  RPC Connections Count: 1
  Network Type: mainnet
  TX Pool Size: 0
  White Peerlist Size: 1000
  Grey Peerlist Size: 4999
  Update Available: false

Three log dumps from this 20 hour long run, the first showing startup behavior, the second after 12 hours, and the last after 20 hours. At the 12 and 20 hours marks, I had obtained 20 blocks each for a total of 40 blocks over 20 hours. bitmonero_startup.log bitmonero_12hrs_tail.log bitmonero_20hrs_tail.log

The network appears healthy (AFAICT) with multiple connected nodes in state "syncronizing"
The usb ssd tests out with 200/230 write/read MB/s and I don't see issues in dmesg
It does see new top block candidates in the logs, it just either doesn't seem to get them, or fails to process them

The SSD is constantly being accessed, here's a top listing, I'm not sure how to get ssd stats?

PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
16975 pinodex+  20   0  761044 310000    556 S  58.6   7.9 737:05.73 monerod

I don't have anything special on my network, just a dd-wrt flashed router that does dhcp and upnp
The blockchain was obtained using the windows monero core wallet "monero-gui-v0.17.3.0" and transferred to the pi via sftp. I have transferred the blockchain three times after syncing it to top with the desktop software. The desktop and the pi monerod are always shut down before and during transfer. The desktop software has no issues with the blockchain and the pi does at least occasionally add new blocks, just at far slower than realtime speed.
It syncs significantly faster if I let it go from scratch, but would take weeks to get the full chain due to the pi's lack of grunt
The usb ssd is formatted as EXT4 and attached to a mount point that holds the blockchain data only
I have tried this across two completely fresh sets of hardware (pi, ssd enclosure, ssd) and across multiple installs to the SD card and blockchain copies
I've started this node without p2pstate.bin using defaults, deleted p2pstate.bin and pointing to a known good node with --seed-node option, and with a copy of my desktop p2pstate.bin (desktop has no issues syncing).

Any idea where I can look to debug further? If I need a new build it'd be a lot easier for me to grab one via GIT if available than to rebuild on the pi. I'm a windows dev without a lot of linux experience but will give it my best if needed.

tcoza commented 2 years ago

On most Linux distros exFAT is supported by a FUSE driver, not an actual in-kernel filesystem driver. So at best it will always be 2-3x slower than a native filesystem. ext4 is of course a native filesystem.

Yeah, that explanation would be more satisfying if monerod didn't run perfectly fine on my Linux laptop on that same exfat SSD but whatever.

Thank you all for helping me out, really appreciate it.

nkinnan commented 2 years ago

Your laptop has a lot more CPU grunt :)

monero-project / monero

unable to obtain and/or process more than 40 new blocks per day #8144