graphprotocol / graph-node

Graph Node indexes data from blockchains such as Ethereum and serves it over GraphQL
https://thegraph.com
Apache License 2.0
2.91k stars 971 forks source link

[Feature] Configure views for subgraphs status data unioned from sharded databases #4825

Open datanexus-vincent opened 1 year ago

datanexus-vincent commented 1 year ago

Description

Derek (@Data-Nexus) had the great idea to create views for the different tables from the subgraphs schema that union all of the different foreign tables available in each shard's schema. For example, we have a subgraph_deployment_vw, subgraph_error_vw , subgraph_manifest_vw, subgraph_version_vw, and subgraph_vw. (Note: we didn't make one for subgraphs.subgraph_deployment_assigment because we found that table to be identical across all shards)

Would it be possible to automatically create views that union each foreign table from the shard_{shard name}_subgraphs schemas? That would make overview grafana dashboards for sharded deployments a lot easier to create and distribute.

Are you aware of any blockers that must be resolved before implementing this feature? If so, which? Link to any relevant GitHub issues.

If my other feature request is implemented, it would be useful to have a union view for public.ethereum_networks as well.

Some information to help us out

madumas commented 1 year ago

these foreign tables may also be required for copy/grafting https://github.com/graphprotocol/graph-node/issues/4719

lutter commented 1 year ago

They are not needed for copy/grafting; graph-node handles that already by picking the right shard/foreign schema as needed.

The problem with these views is in how they interact with updates to the mappings, if they are ever needed: any update to them will drop the views. It'll require some more logic in this already very complicated piece of code. I'll have to think about that a bit.

github-actions[bot] commented 7 months ago

Looks like this issue has been open for 6 months with no activity. Is it still relevant? If not, please remember to close it.

azf20 commented 7 months ago

Any thoughts on this one @lutter?