hirosystems / chainhook

Extract transactions from Stacks and Bitcoin and build event driven re-org resistant indexers and databases.
GNU General Public License v3.0
144 stars 57 forks source link

NFT-event file append pulls back events that are not associated with the asset-identifier #354

Closed timstackblock closed 1 year ago

timstackblock commented 1 year ago

Describe the bug NFT-event file append pulls back events that are not associated with the asset-identifier asset_identifier": "ST1PQHQKV0RJXZFY1DGX8MNSNYVE3VGZJSRTPGZGM.redeeem-nft-v0",

I ran this predicate and only wanted to return mint events for "ST1PQHQKV0RJXZFY1DGX8MNSNYVE3VGZJSRTPGZGM.redeeem-nft-v0"

Here is the predicate file

Screen Shot 2023-07-26 at 10 50 52 AM

I successfully got records for that asset identifier which is great but the file is very large and there are a lot of records for other NFTs that do not mat ch the asset identifier "ST1PQHQKV0RJXZFY1DGX8MNSNYVE3VGZJSRTPGZGM.redeeem-nft-v0",

Here are screen shots this is the GOOD data

Screen Shot 2023-07-26 at 10 52 39 AM

Here is the bad data the event type also seems to not filter for "MINT:" events and is pulling in data that is outside of the scope. You will see transfer data and you will see asset identifiers for "asset_class_identifier":"ST39HFKW38EPPPRQ1R52GK02WCN8314DQAP1P6AYE.test-wn::wolfs-pack-academy" the filtering seems to be broken.

Screen Shot 2023-07-26 at 10 55 39 AM

Please let me know if you need more information

lgalabru commented 1 year ago

Thanks for the time spent on this @timstackblock, I think it's time for us to take on https://github.com/hirosystems/chainhook/issues/287. @timstackblock in future bug reports, could you please provide the predicate in json format, instead of screenshots? Also when reporting the results, providing a copy paste (full block or full transaction) would speed up debugging! Thank you again for your halp 🙏

timstackblock commented 1 year ago

Thanks for the time spent on this @timstackblock, I think it's time for us to take on #287. @timstackblock in future bug reports, could you please provide the predicate in json format, instead of screenshots? Also when reporting the results, providing a copy paste (full block or full transaction) would speed up debugging! Thank you again for your halp 🙏

sure no problem @lgalabru will do

MicaiahReid commented 1 year ago

@timstackblock This issue should be fixed currently in the develop branch. Can you confirm?

timstackblock commented 1 year ago

sure let me take a look

timstackblock commented 1 year ago

Please scroll to the bottom of the explorer page in the link below to see the MINT NFT events chainhooks should be pulling back

https://explorer.hiro.so/txid/ST1PQHQKV0RJXZFY1DGX8MNSNYVE3VGZJSRTPGZGM.redeeem-nft-v0?chain=testnet

Here is the error I am getting when I run the predicate

Screen Shot 2023-08-03 at 12 19 52 PM

Chainhooks seems like it cannot pull back the mint events for the NFT it is getting 0 occurrences but the NFT events are present. There is also a strange new error Aug 03 16:19:25.089 ERRO unable to parse stacks block_header EOF while parsing a string at line 1 column 3616 Please let me know if there is an issue with the NFT I have selected

I have pasted the predicate below so you can run it

{ "chain": "stacks", "uuid": "c3c3bf08-2035-40bf-a5eb-0e4295911f8a", "name": "nft event file", "version": 1, "networks": { "testnet": { "start_block": 20835, "end_block": 38357, "if_this": { "scope": "nft_event", "asset_identifier": "ST1PQHQKV0RJXZFY1DGX8MNSNYVE3VGZJSRTPGZGM.redeeem-nft-v0", "actions": ["mint"] }, "then_that": { "file_append": { "path": "automate/tests/stacks-predicates/nft-event/nft-event-file-result.json" } } } } }

timstackblock commented 1 year ago

Here is the transaction

The first MINT NFT_EVENT is in block 20835 that is why we start the block there and the last one is in 38357 there are many in between

https://explorer.hiro.so/txid/0xf3fd7d7278a104ddafaf4517d371bdbf3d75b8ecdbb4cc4ac51488f4240f76de?chain=testnet

Screen Shot 2023-08-04 at 8 55 47 AM
MicaiahReid commented 1 year ago

Hey @timstackblock, I've found what's going on here. The asset id you are using is deployer_address.contract_name, but it should be deployer_address.contract_name::nft_name. For example, rather than "asset_identifier": "ST1PQHQKV0RJXZFY1DGX8MNSNYVE3VGZJSRTPGZGM.redeeem-nft-v0", it should be "asset_identifier": "ST1PQHQKV0RJXZFY1DGX8MNSNYVE3VGZJSRTPGZGM.redeeem-nft-v0::redeeem-nft-v0"

MicaiahReid commented 1 year ago

This predicate worked for me:

{
  "chain": "stacks",
  "uuid": "c3c3bf08-2035-40bf-a5eb-0e4295911f8a",
  "name": "nft event file",
  "version": 1,
  "networks": {
    "testnet": {
      "start_block": 20835,
      "end_block": 20835,
      "if_this": {
        "scope": "nft_event",
        "asset_identifier": "ST1PQHQKV0RJXZFY1DGX8MNSNYVE3VGZJSRTPGZGM.redeeem-nft-v0::redeeem-nft-v0",
        "actions": ["mint"]
      },
      "then_that": {
        "file_append": {
          "path": "automate/tests/stacks-predicates/nft-event/nft-event-file-result.json"
        }
      }
    }
  }
}
timstackblock commented 1 year ago

Thanks for looking into this. One last question using this updated asset identifier I got 22 occurrences but there are 55 mint function calls in the explorer between 20835 and 38357. Is there additional filtering criteria in there somewhere? It looks like its skipping multiple function calls in the same block, when I look at the output file I see all unique blocks listed but in explorer I can see there are multiple MINT events in the same block and they are not being pulled back.

{
  "chain": "stacks",
  "uuid": "c3c3bf08-2035-40bf-a5eb-0e4295911f8a",
  "name": "nft event file",
  "version": 1,
  "networks": {
    "testnet": {
      "start_block": 20835,
      "end_block": 38357,
      "if_this": {
        "scope": "nft_event",
        "asset_identifier": "ST1PQHQKV0RJXZFY1DGX8MNSNYVE3VGZJSRTPGZGM.redeeem-nft-v0::redeeem-nft-v0",
        "actions": ["mint"]
      },
      "then_that": {
        "file_append": {
          "path": "automate/tests/stacks-predicates/nft-event/nft-event-file-result.json"
        }
      }
    }
  }
}
Screen Shot 2023-08-07 at 10 57 17 AM

If you look at block 20836 only 1 record is pulled back for that block but if you look into the explorer multiple mints occurred in that block, below is a screenshot in explorer of all the mints in the block we are not pulling back.

Screen Shot 2023-08-07 at 11 31 47 AM Screen Shot 2023-08-07 at 11 25 30 AM
MicaiahReid commented 1 year ago

If you look at block 20836 only 1 record is pulled back for that block but if you look into the explorer multiple mints occurred in that block

What are you using to gauge how many records are "pulled back"? As I mentioned in this comment, when an "occurrence" is listed in the Chainhook output, this means that a block has at least on transaction that matches a predicate's criteria. Because of that one match, the whole block will be sent to the output file.

So, for block 20836, since there is at least one event that match the specified predicate, the whole block is output to the file. Inside of that block is all of the transactions in the block with all of that transaction's events. So, that one block counts as one "occurrence", but the data within that occurrence has multiple matching events.

Does that make sense?

Maybe what is confusing is that we output X occurrences found and instead it should be X blocks found that contain transactions or events matching predicate conditions, but that is a little wordy, haha

timstackblock commented 1 year ago

Got it makes sense I was expecting to count events but the block for the event is being pulled back

MicaiahReid commented 1 year ago

Awesome! Closing this one as being fixed by #364!