cosmos / cosmos-sdk

:chains: A Framework for Building High Value Public Blockchains :sparkles:
https://cosmos.network/
Apache License 2.0
6.26k stars 3.62k forks source link

[Bug]: Result and Pagination are not available in gRPC cosmos.base.tendermint.Service. GetValidatorSetByHeight / GetLastestValidatorSet #18054

Closed chemonoworld closed 5 months ago

chemonoworld commented 1 year ago

Is there an existing issue for this?

What happened?

This bug seems to be similar to #8560 but different a little bit.

The pagination "total" in the return result of this gRPC call to cosmos.base.tendermint.Service. GetValidatorSetByHeight is always 180 and the number of elements in Validators is 100 even if a PageRequest is already set in the GetValidatorSetByHeightRequest. The validator set will return maximum of 100 validators without the pagination key to query the next page. The corresponding rest API doesn't work.

Cosmos SDK Version

0.45, 0.47, 0.46(probably, but not checked)

How to reproduce?

Not happened in Tendermint RPC

https://rpc.cosmos.bh.rocks/validators?height=17300000

But it happens in rest API

https://rest.cosmos.directory/cosmoshub/cosmos/base/tendermint/v1beta1/validatorsets/latest
https://rest.cosmos.directory/cosmoshub/validatorsets/latest

It looks good on the face of it but the number of validators is always 100 even if "pagination.limit" in query string is changed.

Also happens in gRPC

grpcurl -plaintext cosmos-grpc.polkachu.com:14990 cosmos.base.tendermint.v1beta1.Service.GetLatestValidatorSet | jq '.pagination'

This always return the below result.

{
  "total": "180"
}

However,

grpcurl -plaintext cosmos-grpc.polkachu.com:14990 cosmos.base.tendermint.v1beta1.Service.GetLatestValidatorSet | jq '.validators | length'

This result in the return of this command is 100.

I also tried these with gRPC code(Golang) changing Request Pagination parameter but can take just only the same result.

lucaslopezf commented 6 months ago

The core issue arises when querying validators from cometBFT, which enforces a maximum query limit of 100 validators to avoid overloading the node. This means that even if you set a limit above 100, 101 or 200, the query will always return the first page capped at 100 validators. You can find the code that enforces this limit here: cometBFT Code

Therefore, to retrieve the second page (the remaining 80 validators), you must use an offset equal to the limit you're setting, which is also the maximum limit allowed.

If you execute the following call:

grpcurl -plaintext -d '{"pagination": {"offset": "100","limit":100}}' cosmos-grpc.polkachu.com:14990 cosmos.base.tendermint.v1beta1.Service.GetLatestValidatorSet | jq '.validators | length'

You will receive the remaining 80 validators as expected.


@tac0turtle it would be great to have this pagination behavior documented. I couldn't find anything about it, do you know if it's documented somewhere? If not, should we add it? Where do you think would be a good place to put it?

educlerici-zondax commented 6 months ago

@tac0turtle can you take a look at @lucaslopezf comment please? 🙏

lucaslopezf commented 6 months ago

Hi @chemonoworld did the answer work?