graphprotocol / graph-node

Graph Node indexes data from blockchains such as Ethereum and serves it over GraphQL
https://thegraph.com
Apache License 2.0
2.91k stars 969 forks source link

Optimize `store_get_asc_new` #3874

Open leoyvens opened 2 years ago

leoyvens commented 2 years ago

This shows as a major time consumer in a few subgraphs including: https://thegraph.com/hosted-service/subgraph/sushi-labs/users-polygon (Qma5YzDtseXGJfwt1dGMmRwMaqvbgdSgnprWSdnp5gQf3n) https://thegraph.com/hosted-service/subgraph/sushi-labs/users-ethereum (QmV4tr3xCx21z77YhPNLM7dJJtGmQ7tvzKpLJTBHNrBd5B) https://thegraph.com/hosted-service/subgraph/sushiswap/sushiswap-polygon (QmT3h6RJvw5zJ6Gq6v6xt1sNExxHcUXEXc4y9rGHjYmPFj)

Since this is purely a CPU operation it should be possible to benchmark it locally and hopefully make significant optimizations.

OlaStenberg commented 2 years ago

This shows as a major time consumer in a few subgraphs including: https://thegraph.com/hosted-service/subgraph/sushi-labs/users-polygon (Qma5YzDtseXGJfwt1dGMmRwMaqvbgdSgnprWSdnp5gQf3n) https://thegraph.com/hosted-service/subgraph/sushi-labs/users-ethereum (QmV4tr3xCx21z77YhPNLM7dJJtGmQ7tvzKpLJTBHNrBd5B) https://thegraph.com/hosted-service/subgraph/sushiswap/sushiswap-polygon (QmT3h6RJvw5zJ6Gq6v6xt1sNExxHcUXEXc4y9rGHjYmPFj)

Since this is purely a CPU operation it should be possible to benchmark it locally and hopefully make significant optimizations.

Great, this is a major blocker for us, we have a release planned and the sushiswap/sushiswap-polygon has only moved 1% in 3 days

azf20 commented 2 years ago

Hey @leoyvens I was looking into this and store_get_asc_new is much less of a contributor to the metrics now for the users subgraph, but it is still a big contributor to QmT3h6RJvw5zJ6Gq6v6xt1sNExxHcUXEXc4y9rGHjYmPFj Looks like the slow handler being triggered is onSync, @OlaStenberg can you point me towards the most recent code to dig into that?

github-actions[bot] commented 1 year ago

Looks like this issue has been open for 6 months with no activity. Is it still relevant? If not, please remember to close it.