search

decred / dcrdata

Decred block explorer, with packages and apps for data collection and storage. Written in Go.
https://dcrdata.decred.org/
ISC License
129 stars 128 forks source link

Node version stats #1214

Open xaur opened 5 years ago

xaur commented 5 years ago

Show various interesting stats about Decred nodes.

First iteration - current stats

Basically match and improve upon this page https://dcred.eu/nodeStats

Automating collection of these stats would be a huge help to Decred Journal's Network section, and interesting public info in general.

Second iteration - historical data

Save hourly snapshots of all items from First iteration, and render them in charts.

Side effect: attract more attention to the problem of the full node count not growing.

chappjc commented 5 years ago

Much like the design of rateserver and the exchanges package, there will need to be a separate crawler data provider. The simplest solution is to build an HTTP API into dcrseeder.

xaur commented 5 years ago

https://dcred.eu/nodeStats is "down for good" while https://dcrstats.com/map shows a lot of red lines, apparently after the recent consensus change.

As of now, we don't have a source for node stats for Decred Journal.

chappjc commented 5 years ago

For Decred Journal, I can pull stats out of a crawler I run. We will put this issue on the roadmap for 5.1

xaur commented 5 years ago

That's great! On the big picture, DJ is not too big of a motivation to implement such a big feature. Imo the big motivation is to have the data available so that anyone can assess network health. With both of those services down or showing stale info we got a bit "blind".

chappjc commented 5 years ago

I posted the following in Matrix. Mirroring here.

decred-public-nodes-2019-06-02.xlsx

public_nodes.json.gz

Service flags: full node = 1 bloom filter support =2 committed filter support = 4 e.g. full node with committed filter support = 5

     0     3     5     7
     4     6   232    28
xaur commented 5 years ago

Thanks, the continent, country, ISP, organization etc data is very interesting and deserves visualization (after some grouping and cleanup).

For user agents, I found this chart that illustrates what I would like to see:

https://charts.dcr.farm/d/000000014/nodes?orgId=1&from=now-40d&to=now

It shows charts of total listening nodes and histogram of user agents over time, which is very helpful to see how the network is upgrading.

dcr-farm-nodes

noahpierau commented 4 years ago

Is someone actively working on this? It's an important feature to have.

chappjc commented 4 years ago

Not that I'm aware of. However, I think the first steps in this feature are to expanddcrseeder's API since dcrdata will not actually do the crawling.

raedah commented 4 years ago

@ademuanthony has done some work on this in the dcrextdata code base

@noahpierau can you share more specifically what you find important about it?

ghost commented 4 years ago

We need the NodeMap like https://bitnodes.earn.com/

noahpierau commented 4 years ago

It makes the global nature of Decred visible to outsiders. Showing a map with nodes all over the world is great for conferences. Also, a node map can be used to monitor how decentralized the network is.

xaur commented 3 years ago

charts.dcr.farm has been discontinued and now we don't have a source of month's average listening/total node counts for DJ.

dcrstats.com/map is the only working nodes map I know, but it shows a very short list of nodes.

Unless there is another nodes dashboard somewhere, we miss an important network metric.

xaur commented 3 years ago

Phew, we have something here: https://dcrextdata.planetdecred.org/nodes

xaur commented 3 years ago

Another new tool showing node count, world map and top 10 by country and ISP

https://nodes.jholdstock.uk/

Thanks @jholdstock!

xaur commented 3 years ago

I missed it initially but it also shows node version counts and has an epic theme!

https://nodes.jholdstock.uk/user_agents?theme=386