celestiaorg / celestia-node

Celestia Data Availability Nodes
Apache License 2.0
919 stars 917 forks source link

bug(discovery): discovery is unable to find all full nodes via dht lookup #3573

Open walldiss opened 1 month ago

walldiss commented 1 month ago

Implementation ideas

I've conducted multiple tests to gather detailed information on the number of full nodes (FN) currently active within the network. The results indicate that the actual number of active nodes is significantly lower than expected. Below are the details of the nodes observed during the tests on both the Mocha and Mainnet environments:

Mocha Network:

Full Nodes: 54 Archival Nodes: 12

Mainnet:

Full Nodes: 17 Archival Nodes: 1

This discrepancy suggests there may be an underlying issue related to querying or persisting DHT records. It's crucial to investigate whether this is an isolated incident or a widespread problem affecting the network's reliability and data accessibility.

walldiss commented 1 month ago

Additionally, we have shrexSub, that also monitors amount of full nodes within active connections. Mainnet metrics for 3 nodes suggests that there are 100+ FN connected for each reporting node.

image