Gain ownership of the nodes' telemetry data with Near One.
Currently, the telemetry gathering endpoint and logic behind it are implemented under near-explorer, which is officially sunset and will not be continued soon.
The database containing telemetry data is created on the same instance as ExplorerDB. The latter is constantly under stress, which is the main reason for the NEAR Explorer project's deprecation.
We at Near One / Protocol require telemetry data for different reasons, and it's not up to me to tell you that. Currently, there is no official owner of that data and its pipeline, causing much trouble during issues or outages.
Example: @gmilescu contacted me because the mainnet telemetry database is filled with data from statelessnet. The root cause of the problem is that the nodes from the statelessnet have https://explorer.mainnet.near.org/api/nodes by default in the config.json. Since a change of the config parameter on the decentralized node that we don't run is barely possible, we cannot solve this problem.
Links to external documentations and discussions
N/A
Estimated effort
0.25-0.5 HC for the technical part (steps: 1, 2, 4, 5) and who knows how much time for the "politics" part (steps: 3, 6
Spin up the PostgreSQL database under Near One "jurisdiction" for the telemetry data
Define/implement the simple endpoint for telemetry to be pushed (it might be a standalone service or even an entire NEAR Explorer if necessary)
Work with Pagoda's SRE team to proxy the existing endpoints to our service from step 2
(optional) Change the default config.json we publish to have our new endpoint
(optional) Change the default value for config.json we generate running neard init
Bridge the access to the telemetry data owned by us to Pagoda
Assumptions
This initiative assumes that the telemetry data is crucial for Near One. Otherwise, it doesn't make sense to invest in it.
Pre-requisites
Near One GCP access
Dedicated domain(s) for the telemetry service
(preferable) Near One Grafana to set pg-connector to visualize the telemetry data on dashboards
Goals
Gain ownership of the nodes' telemetry data with Near One.
Currently, the telemetry gathering endpoint and logic behind it are implemented under
near-explorer, which is officially sunset and will not be continued soon.The database containing telemetry data is created on the same instance as ExplorerDB. The latter is constantly under stress, which is the main reason for the NEAR Explorer project's deprecation.
We at Near One / Protocol require telemetry data for different reasons, and it's not up to me to tell you that. Currently, there is no official owner of that data and its pipeline, causing much trouble during issues or outages.
Example: @gmilescu contacted me because the
mainnettelemetry database is filled with data fromstatelessnet. The root cause of the problem is that the nodes from thestatelessnethavehttps://explorer.mainnet.near.org/api/nodesby default in theconfig.json. Since a change of the config parameter on the decentralized node that we don't run is barely possible, we cannot solve this problem.Links to external documentations and discussions
N/A
Estimated effort
0.25-0.5 HC for the technical part (steps: 1, 2, 4, 5) and who knows how much time for the "politics" part (steps: 3, 6
config.jsonwe publish to have our new endpointconfig.jsonwe generate runningneard initAssumptions
This initiative assumes that the telemetry data is crucial for Near One. Otherwise, it doesn't make sense to invest in it.
Pre-requisites
Out of scope