ls-miners and ls-sharders only return first 20

[sculptex]

Due to pagination on SDK only first 20 are returned, CLI should either combine the multiple offset pages and return combined list or add offset or pagination to obtain full list.

[K-Kumar-01]

Hey @sculptex Was trying to work on the issue, but was facing some difficulties. I was unable to find the pagination on SDK

if err = zcncore.GetMiners(cb); err != nil {
            log.Fatal(err)
        }

The function is used for fetching all miners. However, when i went deep into GetMiners it called https://pkg.go.dev/github.com/0chain/gosdk/zcncore@v1.8.11-0.20221213120149-c3bbc5ed747d#GetInfoFromSharders Here also, I did not see any limit or offset arguments.

Am i doing anything wrong here?

[sculptex]

@K-Kumar-01 I observer limit on /v1/screst/6dba10422e368813802877a85039d3985d96760ed844092319743fb3a76712d9/getMinerList

it only returns 20 results. I paginate next 20 using ?offset= until empty result but not checked through gosdk how it is accessed

[K-Kumar-01]

/v1/screst/6dba10422e368813802877a85039d3985d96760ed844092319743fb3a76712d9/getMinerList

Yeah, offset seems to work here, though I am only getting 3 results for now. But in gosdk, I wasn't able to find the limit and offset parameters, unlike the getBlobbers() Link here

[boddumanohar]

@K-Kumar-01 you are on the correct path,

as you see in gosdk GetMiners function,

https://github.com/0chain/gosdk/blob/4bd780ec873c340fa8c6b6b7925d5c396d48be97/zcncore/wallet_base.go#L814-L822

we makes a call to /getMinerList endpoint on sharders

https://github.com/0chain/0chain/blob/5e1e5c1189eb3bba7d67cb01878a192e0fc3f6f7/code/go/0chain.net/smartcontract/minersc/handler.go#L685-L712

to the URL that is passed here, you might have to append pagination query parameters

So you might have to add a new function or update the existing in gosdk that does pagination on this endpoint and raise a seperate PR to gosdk repo and then an other PR with appropriate changes for pagination in zwalletcli.

[sculptex]

@K-Kumar-01

you can test with beta pre-mainnet network (beta.zus.network/dns) which has 53 miners registered

https://premainnet.zus.network/sharder01/v1/screst/6dba10422e368813802877a85039d3985d96760ed844092319743fb3a76712d9/getMinerList

[K-Kumar-01]

@sculptex @boddumanohar

Regarding the issue would like to point my work proposal.

Implementation for miners

if err = zcncore.GetMiners(cb); err != nil {
            log.Fatal(err)
        }

where

// GetMiners obtains list of all active miners.
func GetMiners(cb GetInfoCallback) (err error) {
    if err = CheckConfig(); err != nil {
        return
    }
    var url = GET_MINERSC_MINERS
    go GetInfoFromSharders(url, 0, cb)
    return
}

for blobbers

zcncore.GetBlobbers(statusBar, limit, offset, !active)

where

func GetBlobbers(cb GetInfoCallback, limit, offset int, active bool) {
    getBlobbersInternal(cb, active, limit, offset)
}

func getBlobbersInternal(cb GetInfoCallback, active bool, limit, offset int) {
    if err := CheckConfig(); err != nil {
        return
    }

    var url = withParams(STORAGESC_GET_BLOBBERS, Params{
        "active": strconv.FormatBool(active),
        "offset": strconv.FormatInt(int64(offset), 10),
        "limit":  strconv.FormatInt(int64(limit), 10),
    })

    go GetInfoFromSharders(url, OpStorageSCGetBlobbers, cb)

    return
}

I am thinking of extending a similar thing to both miners and sharders. Let me know your thoughts.

This would require a change first in gosdk then zwalletcli as stated by @boddumanohar

[boddumanohar]

@K-Kumar-01 Great, it should work fine, go ahead. 👍

[K-Kumar-01]

@boddumanohar @sculptex Can you please take a look at PR here