Closed TPXP closed 1 week ago
thanks @TPXP , can you clarify what happens when tzkt returns more delegators than numDelegators? Who are the extra delegators in this case? Are they duplicates? what's happening with the payouts? Is the program crashing?
Hello @nicolasochem , thanks for the reply.
When TzKT replies more delegators, the len(res["delegators"]) == res["numDelegators"]
condition never triggers so we make more and more calls with increasing offset. TRD ends up failing to calculate rewards for the cycle with the following exception.
producer - ERROR - tzkt error at payment producer loop: 'Max sequent calls number exceeded (256)', will try again.
I couldn't find any duplicates in the example I'm giving - maybe a few weird addresses but that's all. It seems that numDelegators
is just wrong. Here's how I checked:
$ wget https://api.tzkt.io/v1/rewards/split/tz3LV9aGKHDnAZHCtC9SjNtTrKRu678FqSki/751\?offset\=0\&limit\=10000 -O rewards.json
$ jq '.numDelegators' ./rewards.json
3224
$ jq '.delegators[].address' ./rewards.json | sort | uniq | wc -l
3225
$ jq '.delegators[].address' ./rewards.json | sort | less
"KT1JBkpjozcKXWpHJLVhU67ZpXFhzH9Hekzg"
"tz1KeBQPzzGA7rBDcUY7biCKEKTSnxX291Hm"
...
"tz1iyXtwz3bKa6r8KEh6CnQU6CayU4msz4zj"
"tz2VJ6pJnnb9rYHFSQzUiZHX1M3uGnvGu4pz"
# Maybe 0 balances? (No)
$ jq '.delegators[].delegatedBalance' ./rewards.json | sort -h | less
154
[...]
6025854993128
@Groxan can you please check on your end why this would be happening?
That indeed may happen, when an account re-delegates to another baker, but still has unstake requests with the previous baker. I.e. the account has delegated balance with two different bakers at the same time, but counted as delegator only with the actual one.
In other words, numDelegators
doesn't include those who actually delegated to a different baker, but still has pending unstake requests.
The best way to check for end of data is response.length < limit
.
Thanks for confirming @Groxan , I've updated the PR to fetch pages until we get less delegators than limit
.
thanks everyone
name: fix: tzkt api may return more delegators than numDelegators about: Create a pull request to make a contribution labels: bugfix, tzkt_api
This PR resolves the issue. The following steps were performed:
Analysis: TZTK seems to return more delegators than
numDelegators
in some cases on therewards/split
endpoint. Handle this gracefully and move on. Seetz3LV9aGKHDnAZHCtC9SjNtTrKRu678FqSki
at cycle 751 for an example: https://api.tzkt.io/v1/rewards/split/tz3LV9aGKHDnAZHCtC9SjNtTrKRu678FqSki/751?offset=0&limit=10000Solution: Update the length condition to stop if we fetched at least as much delegators as
numDelegators
. Also, add another safeguard by stopping the loop if we're not fetching any more validators (indicating we've reached the end of the list) - this way we will stop if TzKT returns less validators thannumDelegators
.Implementation: Rough description/explanation of the implementation choices.
Performed tests: Run TRD for baker
tz3LV9aGKHDnAZHCtC9SjNtTrKRu678FqSki
for cycle 751, confirm it worksDocumentation: N/A
Check list:
[x] I tested my changes and confirm they fix the issue I was facing
Work effort: 2 hours
Possibly fixes https://github.com/tezos-reward-distributor-organization/tezos-reward-distributor/issues/616