I was playing around and came up with a command to count the holders of a token. I use 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 in the exemple. I also directly use the Transfer topic instead of using articulate and adding filters.
This makes a list of unique receiver of the token, in a single line.
chifra export 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 --logs 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef --fmt csv --no_header | cut -d "," -f 11 | sed 's/0x000000000000000000000000/0x/' | sort | uniq | tr '\n' ' '
I then use this line in a chifra tokens command to get the balances of these wallets, filter only the positive ones, and finally count the number of results:
chifra tokens --fmt csv 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 $(chifra export 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 --logs 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef --fmt csv --no_header | cut -d "," -f 11 | sed 's/0x000000000000000000000000/0x/' | sort | uniq | tr '\n' ' ') --no_header | awk -F, '$7 > 0 {print $2}' | wc -l
[3:15 PM]john: In a few hours timeframe, I got 14927, 14919 or 14975 holders. DexScreeer says 14,985, Etherscan says 14,980. I have no clue about how etherscan gets it, I don't think it's in real time. They also indicate 199,849 token transfers, while I get 199,774 using chifra
chifra export 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 --logs --fmt csv --cache --no_header 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef --emitter 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 | wc -l
Would you have any information about where these differences can come from? I'm particularly curious about etherscan showing a higher number of Transfers
[8:11 PM]dreadedhamish: I have no answers, but I'd be curious about the differences - so which wallets do they disagree about. Maybe then check the balance of those addresses through the ERC20 contract?
[11:42 PM]john: Actually, computing the results (amount sent vs received) from the logs list from chifra (the 1st call I shared above) is wayy faster than querying balances for each wallet. But it was fun to wrap it all in one line. And this gives me currently 14,978 holders vs 15,002 on etherscan. Etherscan does not provide the list (it’s a pro api endpoint) so I think I won’t be able to further compare. Total transfers is still a little above too
[11:59 PM]john: I noticed that the receipts from chifra export do not contain all the logs of a transaction. I guess this is normal as we only get those where the address appears, even though I was initially expecting to have all the transaction data.
I think I have a workaround for my use case, but I’d be interested in any suggestions on how to get all the tx with all their logs for an address to better understand trueblocks. Maybe traces, but that would be slower ?
[12:07 AM]dreadedhamish: we need someone with a pro API key.
[4:49 AM]dawid | TrueBlocks.io: I think that the differences may be caused by TrueBlocks waiting 28 blocks before "staging" appearances (in this case "staging" means "I can query it"). If I'm not mistaken it's around 15 minutes on mainnet. Etherscan seems to wait only 1 minute. We wait longer to be sure no reorg happens after we staged appearances.
[4:50 AM]dawid | TrueBlocks.io: That's interesting. Could you share transaction hash?
[8:38 AM]john: Thanks, I didn’t know about that
[8:45 AM]dreadedhamish: so could you grab the total and blocknumber from etherscan, when wait a while and get holders at that block from chifra?
[9:20 AM]john: chifra transactions do get all logs. But for exemple, we can get the transaction 0x9c095778f4945dfefb0b194f734ffd85b7a1345de06d8957c9c6b749a21a546c by running chifra export 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 --receipts --fmt json --first_block 200209341 --max_records 1. I use a token contract address because this is what I’m focused on.
This tx is a uniswap swap between two tokens (using two uniV2 pairs), and has 10 event logs on etherscan. But only the 3 token transfer events corresponding to the token in the command are shown in the result of a chifra export [addrs] command. The same goes for every swaps.
I was then curious about doing a search from a wallet address instead of a token. Getting the same transaction using the sender address:
chifra export 0x76dD80dA9F7FAa619c251D5AC8E9011F4f9185D4 --receipts --fmt json --first_block 200209341 --max_records 1
What’s interesting is that we still don’t get all the logs. I have 3 of them: the token approve, and the 1st and last token transfers, because the wallet address is included in these logs topics. Events about the uniswap pools, router, and weth contract are not returned.
[9:41 AM]john: Now I have to new questions:
How does trueblocks and chifra serve handle parallel requests, do we lose performance running multiple queries simultaneously?
Are log details only displayed when using json format? (for chifra export --receipts)
I was playing around and came up with a command to count the holders of a token. I use 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 in the exemple. I also directly use the Transfer topic instead of using articulate and adding filters.
chifra export 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 --logs 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef --fmt csv --no_header | cut -d "," -f 11
gives me the 'to' of each Transfer.
This makes a list of unique receiver of the token, in a single line. chifra export 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 --logs 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef --fmt csv --no_header | cut -d "," -f 11 | sed 's/0x000000000000000000000000/0x/' | sort | uniq | tr '\n' ' '
I then use this line in a chifra tokens command to get the balances of these wallets, filter only the positive ones, and finally count the number of results: chifra tokens --fmt csv 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 $(chifra export 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 --logs 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef --fmt csv --no_header | cut -d "," -f 11 | sed 's/0x000000000000000000000000/0x/' | sort | uniq | tr '\n' ' ') --no_header | awk -F, '$7 > 0 {print $2}' | wc -l
[3:15 PM]john: In a few hours timeframe, I got 14927, 14919 or 14975 holders. DexScreeer says 14,985, Etherscan says 14,980. I have no clue about how etherscan gets it, I don't think it's in real time. They also indicate 199,849 token transfers, while I get 199,774 using chifra chifra export 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 --logs --fmt csv --cache --no_header 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef --emitter 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 | wc -l
Would you have any information about where these differences can come from? I'm particularly curious about etherscan showing a higher number of Transfers [8:11 PM]dreadedhamish: I have no answers, but I'd be curious about the differences - so which wallets do they disagree about. Maybe then check the balance of those addresses through the ERC20 contract? [11:42 PM]john: Actually, computing the results (amount sent vs received) from the logs list from chifra (the 1st call I shared above) is wayy faster than querying balances for each wallet. But it was fun to wrap it all in one line. And this gives me currently 14,978 holders vs 15,002 on etherscan. Etherscan does not provide the list (it’s a pro api endpoint) so I think I won’t be able to further compare. Total transfers is still a little above too [11:59 PM]john: I noticed that the receipts from chifra export do not contain all the logs of a transaction. I guess this is normal as we only get those where the address appears, even though I was initially expecting to have all the transaction data. I think I have a workaround for my use case, but I’d be interested in any suggestions on how to get all the tx with all their logs for an address to better understand trueblocks. Maybe traces, but that would be slower ? [12:07 AM]dreadedhamish: we need someone with a pro API key. [4:49 AM]dawid | TrueBlocks.io: I think that the differences may be caused by TrueBlocks waiting 28 blocks before "staging" appearances (in this case "staging" means "I can query it"). If I'm not mistaken it's around 15 minutes on mainnet. Etherscan seems to wait only 1 minute. We wait longer to be sure no reorg happens after we staged appearances. [4:50 AM]dawid | TrueBlocks.io: That's interesting. Could you share transaction hash? [8:38 AM]john: Thanks, I didn’t know about that [8:45 AM]dreadedhamish: so could you grab the total and blocknumber from etherscan, when wait a while and get holders at that block from chifra? [9:20 AM]john: chifra transactions do get all logs. But for exemple, we can get the transaction 0x9c095778f4945dfefb0b194f734ffd85b7a1345de06d8957c9c6b749a21a546c by running chifra export 0x3ffeea07a27fab7ad1df5297fa75e77a43cb5790 --receipts --fmt json --first_block 200209341 --max_records 1. I use a token contract address because this is what I’m focused on. This tx is a uniswap swap between two tokens (using two uniV2 pairs), and has 10 event logs on etherscan. But only the 3 token transfer events corresponding to the token in the command are shown in the result of a chifra export [addrs] command. The same goes for every swaps. I was then curious about doing a search from a wallet address instead of a token. Getting the same transaction using the sender address: chifra export 0x76dD80dA9F7FAa619c251D5AC8E9011F4f9185D4 --receipts --fmt json --first_block 200209341 --max_records 1 What’s interesting is that we still don’t get all the logs. I have 3 of them: the token approve, and the 1st and last token transfers, because the wallet address is included in these logs topics. Events about the uniswap pools, router, and weth contract are not returned. [9:41 AM]john: Now I have to new questions: How does trueblocks and chifra serve handle parallel requests, do we lose performance running multiple queries simultaneously? Are log details only displayed when using json format? (for chifra export --receipts)