search

graphprotocol / indexer

Graph Protocol indexer components and infrastructure
MIT License
240 stars 132 forks source link

Change the naming scheme for deployments #574

Open lutter opened 1 year ago

lutter commented 1 year ago

Right now, indexer-agent deploys subgraphs under the name indexer-agent/<hash> into graph-node. That makes it impossible to use some graph-node features, especially deployment rules to control how deployments are indexed.

The naming scheme should be changed to <subgraph name>/<hash> so that a subgraph might be called user/name/hash. That would make it possible to use name-based rules with graph-node, e.g., to deploy subgraphs that are known to get a lot of queries into a more powerful database shard.

Edited by @tilacog:

tilacog commented 1 year ago

Reopening this issue as the changes were reverted by #644

fordN commented 1 year ago

Is this still needing to be done @tilacog ?

tilacog commented 1 year ago

Positive. We tried this on #615 but it has been reverted on #644.

datanexus-vincent commented 1 year ago

That would make it possible to use name-based rules with graph-node, e.g., to deploy subgraphs that are known to get a lot of queries into a more powerful database shard.

What is the best workflow for this? From what I can tell your best bet is changing the regex match in the graph-node config toml every time you want to deploy a new subgraph to a VIP shard.

I ask because the graph-node docs currently point towards a slightly different workflow. Here it shows a VIP shard matching the words vip or important. Given these are some of the only docs for sharding, we started architecting our sharded graph-node database with this workflow in mind, and tried manually deploying subgraphs with the naming scheme vip/hash to get the deployments on the right shard.

Ultimately, I think indexers would rather have this behavior be connected to the commands you enter to start syncing a subgraph, like graph indexer rules prepare subgraph_hash [user_defined_keyword], along with the naming convention being something like [user_defined_keyword/]name/hash, than a graph-node config file change. But let me know if I'm misunderstanding the workflow being proposed here.

Side note - with this feature, is it possible to add a check to see if a subgraph has already been deployed under a different name before indexer-agent deploys it under whatever naming convention is chosen? I think that would fix the 'duplicate named deployments pointing to the same schema' bug that has been seen a few times, including here when we added subgraphs to the offchain sync list after manually deploying them with vip keywords in the name.