a16z / nft-analyst-starter-pack

https://a16z.com/2022/03/18/nft-starter-pack-analyze-data-metadata-build-tools/
GNU Affero General Public License v3.0
461 stars 92 forks source link

Support for NFT events? #21

Open vintrocode opened 2 years ago

vintrocode commented 2 years ago

A NFT contract I was trying to pull data for was mysteriously missing mint transfers from the 0 addr. It turned out that the mints were triggered from a separate minting module contract. This minting module contract called the mint function in the actual NFT contract after a series of checks.

The actual NFT contract had transactions only for transfers that happened after minting. This was consistent with the data I pulled and what the transactions tab was showing on Etherscan. However, the events tab on the token showed all the transfers from the 0 addr and I was able to query those using checkthechain.

The minting module contract had transactions for the mints but the nft-analyst-starter-pack was unable to detect erc721 transfers (see traceback at the bottom).

Here's the minting module and the nft contract I'm referring to.

Questions:

  1. Am I right in understanding that this utility only pulls transfers that show up as transactions?
  2. Might there be a way to pull transfer events omitted from the contracts we refer to in order to ensure high coverage? My understanding is this smart contract pattern is somewhat common (mints coming from minting module, NFT contract separate).
nft-analyst-starter-pack git:(main) ✗ poetry run python export_data.py -a $ALCHEMY_API_KEY -c 0xe200dd50001311e65282d19849e9a93c5b37f1ce
Process started for contract address: 0xe200dD50001311e65282d19849e9A93C5B37F1ce
Checking update logs for most recent block...
No existing data. Contract 0xe200dD50001311e65282d19849e9A93C5B37F1ce appears to have been deployed at block 15391107
Exporting token transfers...
No ERC-721 transfers were identified. Therefore, searching for and extracting any ERC-1155 transfers.
Exporting 1155 transfers...
Exporting logs...
Updating block-to-date mapping...
Updating ETH prices...
Traceback (most recent call last):
  File "/Users/vintrocode/work/passage/pascal-the-chain/nft-analyst-starter-pack/export_data.py", line 225, in <module>
    export_data()
  File "/Users/vintrocode/Library/Caches/pypoetry/virtualenvs/nft-analyst-starter-pack-W3ad-lpQ-py3.9/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/vintrocode/Library/Caches/pypoetry/virtualenvs/nft-analyst-starter-pack-W3ad-lpQ-py3.9/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/vintrocode/Library/Caches/pypoetry/virtualenvs/nft-analyst-starter-pack-W3ad-lpQ-py3.9/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/vintrocode/Library/Caches/pypoetry/virtualenvs/nft-analyst-starter-pack-W3ad-lpQ-py3.9/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/vintrocode/work/passage/pascal-the-chain/nft-analyst-starter-pack/export_data.py", line 176, in export_data
    generate_sales_output(
  File "/Users/vintrocode/work/passage/pascal-the-chain/nft-analyst-starter-pack/core/generate_sales_output.py", line 83, in generate_sales_output
    logs_df = pd.read_csv(logs_file)
  File "/Users/vintrocode/Library/Caches/pypoetry/virtualenvs/nft-analyst-starter-pack-W3ad-lpQ-py3.9/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/Users/vintrocode/Library/Caches/pypoetry/virtualenvs/nft-analyst-starter-pack-W3ad-lpQ-py3.9/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 680, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/Users/vintrocode/Library/Caches/pypoetry/virtualenvs/nft-analyst-starter-pack-W3ad-lpQ-py3.9/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 575, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/Users/vintrocode/Library/Caches/pypoetry/virtualenvs/nft-analyst-starter-pack-W3ad-lpQ-py3.9/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 933, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "/Users/vintrocode/Library/Caches/pypoetry/virtualenvs/nft-analyst-starter-pack-W3ad-lpQ-py3.9/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1235, in _make_engine
    return mapping[engine](f, **self.options)
  File "/Users/vintrocode/Library/Caches/pypoetry/virtualenvs/nft-analyst-starter-pack-W3ad-lpQ-py3.9/lib/python3.9/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 75, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 551, in pandas._libs.parsers.TextReader.__cinit__
pandas.errors.EmptyDataError: No columns to parse from file
dmatsuoka commented 2 years ago

@vintrocode thanks for the feedback! If I'm understanding your question correctly, here's how it works...

The nft-analyst-starter-pack generates a transfers.csv file based on event logs. This means that transfers are associated with the events emitted by whichever contract address you provide. The "to_address" and "from_address" are associated with the NFT transfer log itself, irrespective of which address triggered the transaction or the contract it initially interacts with.

In your example, transfers from 0x00... (i.e., mints) will show up when running the script on the NFT contract address. The minting module contract is not emitting any transfer logs, so they won't show up there.

For example, this transaction was sent by 0x1d0 to the minting module contract at 0xe20. However, the NFT contract at 0xFd0 is where the mint actually happened, with 0x000... as the "from_address" and 0x1d0 as the "to_address".

I suspect that the data you are interested in analyzing will come from the NFT contract, which means you can use the 0xFd09eb152263488Dc5E4654D9F91F0aEbeE45423 address on this tool to fetch all the data - including mints.