graphprotocol / graph-node

Graph Node indexes data from blockchains such as Ethereum and serves it over GraphQL
https://thegraph.com
Apache License 2.0
2.89k stars 962 forks source link

Encountering a consistent indexing error on Matic "Found no transaction for event" #2552

Closed TravelingTechGuy closed 2 years ago

TravelingTechGuy commented 3 years ago
  1. This is a bug report
  2. I'm trying to deploy my subgraph. It used the ABI of the USDT contract, and is supposed to index an entity whenever the to in the Transfer event is the address of our contract (i.e. whenever someone staked USDT in the contract).

This works well on Ethereum. On Matic (Polygon), we get the following fatal error:

"failed to process trigger: block #15167232 (0x7b2a…0d6b), transaction 4641fb604c8b6b9d515f57f364e8039fb600ed0471a7dcb7f16f2bd1314501f1: Found no transaction for event"

This is consistent, and reproducible. I tried adding a function that leaves the handler in case: the block number is 15167232, the block number is between 15167200 and 15167240 (thought a range would help), if event.transaction is empty - to no avail, this error still happens. My guess is this happens before my handler code, and therefore, not under my control.

My workaround is to add a higher startBlock number to the yaml, but the we lose tons of events.

My expected behavior is either a resolution that allows me to index the graph, or a way to bypass whatever problem triggers the fatal error.

More details:

  1. The subgraph can be found at: https://thegraph.com/explorer/subgraph/travelingtechguy/cvipolygon
  2. I can provide a link to the code, if needed. It has other entities that index well.
miguelmota commented 3 years ago

Sync error - block #16797184

{"data":{"indexingStatusForPendingVersion":{"fatalError":{"message":"failed to process trigger: block #16797184 (0x1d93…909a), transaction df952e2b5bb59beb5acbced5be788277312ec838142a808fcdbe437cff8ff12f: Found no transaction for event"},"nonFatalErrors":[],"subgraph":"QmfDg1oHAPb7TPuy882BfBFZrJKMPNEuib98Em7rFds2nb"}}}

@tilacog

shanefontaine commented 3 years ago

Hey @tilacog

Messages on Polygon that are sent from L1 appear as events that are emitted from the 0x0 address. For example, here is a "transaction" that has an event from one of our users that sent a message from L1 -> L2. This message emitted an event (topic 0x320958176930804eb66c2343c7343fc0367dc16249590c0f195783bee199d094) on L2 that you can see in this transaction.

On any other chain, I would expect this event to be emitted on the transaction that actually updated state. However, on Polygon, the transaction that emits the event is not the transaction that updates state. Might this be the issue that the indexer is running into?

miguelmota commented 3 years ago

Sync error - block #16800192

{"data":{"indexingStatusForPendingVersion":{"fatalError":{"message":"failed to process trigger: block #16800192 (0x131d…5b30), transaction 18ad7854383bfb6bf2712aa95b710ffc254ce91e882c3ed6306f929322dcdd53: Found no transaction for event"},"nonFatalErrors":[],"subgraph":"QmfDg1oHAPb7TPuy882BfBFZrJKMPNEuib98Em7rFds2nb"}}}

@tilacog

schmidsi commented 3 years ago

Hey @miguelmota and others here: We cleaned our db and apparently the new Bor version should fix our problem. Your subgraph is syncing. We hope that we now finally have a stable situation. Let us know if there are any problems.

miguelmota commented 2 years ago

Thanks for the update @schmidsi! Have not seen any syncing errors since the update 🎉

schmidsi commented 2 years ago

Nice. I close this issue then. Feel free to reopen if the problem still persists.

schmidsi commented 2 years ago

Seems like we have this problem now on Mumbai: https://thegraph.com/legacy-explorer/subgraph/lemonadesocial/lemonade-marketplace?version=pending

schmidsi commented 2 years ago

https://thegraph.com/legacy-explorer/subgraph/lemonadesocial/lemonade-marketplace?version=pending failed again with error: "failed to process trigger: block #18160704 (0xfa5e…0628), transaction ff72d1f7e52a138fa8ff82190215119118b7a66f16844eadca4d0e053cf2f6b2: Found no transaction for event"

azf20 commented 2 years ago

Thanks. This is strange, as this was one of the blocks which was cleared from the cache, can you confirm @tilacog ?

tilacog commented 2 years ago

Yes, I can confirm this block was previously deleted.

schmidsi commented 2 years ago

Again 🤯: https://thegraph.com/legacy-explorer/subgraph/lemonadesocial/lemonade-marketplace?version=pending

failed to process trigger: block #18160704 (0xfa5e…0628), transaction ff72d1f7e52a138fa8ff82190215119118b7a66f16844eadca4d0e053cf2f6b2: Found no transaction for event

ChrisLahaye commented 2 years ago

@tilacog could you please help us out with https://thegraph.com/legacy-explorer/subgraph/lemonadesocial/lemonade-marketplace?version=pending

failed to process trigger: block #18223680 (0xd0aa…7fee), transaction 7651a8dca5c6ece8758e46aabf29daddc3fdd504dded7d6cde0f0c9f29ff5d26: Found no transaction for event

Thank you

steve-ng commented 2 years ago

hello @tilacog we noticed this on self-hosted graph node as well, is there an easy way to resync a block?

tilacog commented 2 years ago

Currently, to resync a faulty block you must delete the block record from its database table and restart graph-node afterwards.

piavgh commented 2 years ago

@tilacog : which table are you referring to?

tilacog commented 2 years ago

@piavgh It depends on how graph-node is configured.

It can be either the ethereum_blocks table in primary or the chainXX.blocks table in a dedicated shard.

The chainXX is the Postgres schema identifier for a given network and can be found by querying the public.chains table.

piavgh commented 2 years ago

@tilacog : thanks, I will check to see if I can delete the error block

Btw, why the ethereum_blocks table is so small? I'm indexing the Cronos chain (more than 1 million block), but this table is only about 8 bytes?

photo_2022-01-18 09 14 40

tilacog commented 2 years ago

@piavgh Then I believe your block cache should be in a dedicated table, as in the chainXX case I've mentioned above.

You can check your config file to discover where those blocks are being kept. You can also select the table public.chains to discover the same.

vietthang207 commented 2 years ago

@tilacog : thanks, I will check to see if I can delete the error block

Btw, why the ethereum_blocks table is so small? I'm indexing the Cronos chain (more than 1 million block), but this table is only about 8 bytes?

photo_2022-01-18 09 14 40

The actual table is in a different namespace. You can use \dn to list the namespace, and query the table in the other namespace.

azf20 commented 2 years ago

Closing this for now, please re-open if you see this issue again on Polygon (Matic)

OscBacon commented 1 year ago

Hi, seems like Mumbai still has this issue, on block 18122880. The issue again is on a transaction from the zero address. Note, I am using the hosted service

Subgraph failed with non-deterministic error: failed to process trigger: block #18122880 (0xcc82…42d3), transaction 553d80020c38eae2002b5063c264fe9b7c0b5c422859c5da9df221d40caa2f50: Found no transaction for event, retry_delay_s: 3060, attempt: 9
gssvv commented 1 year ago

Still experiencing this issue with zkSync Era mainnet:

Error: failed to process trigger: block #9376171 (0x01fa…404a), transaction 0000000000000000000000000000000000000000000000000000000000000000: Found no transaction for event
incrypto32 commented 1 year ago

@gssvv can you share the deployment ID of the subgraph ? also that block doesnt seem to exist on zksync era

gssvv commented 1 year ago

@incrypto32 Sure, it's QmW5rbKe8iq98iJD2f2D4CRqkLecu3qZpbkH7hYFvTshkp

KolevDarko commented 11 months ago

I'm also facing the same thing issue, but was curious if this only happens when indexing tokens. @gssvv are you indexing a token contract as well?

xsteadybcgo commented 4 months ago

The same issue with @gssvv , any workaround here?