As a chaincode client, I would like to page through chaincode private data query results

denyeart commented 1 year ago

Current Status

Public data queries can be paginated with bookmark and pagesize using these chaincode APIs: GetStateByRangeWithPagination() GetQueryResultWithPagination() GetStateByPartialCompositeKeyWithPagination()

Similar chaincode APIs are not yet available for private data collections.

Expected

Chaincode client can pass pagesize and bookmark parameters to private data range, composite key, and rich query shim APIs in order to drive pagination.

Solution

Add corresponding chaincode APIs for private data.

Please let us know if you plan to work on this.

No.

denyeart commented 1 year ago

See prior discussion and workaround in Jira at https://jira.hyperledger.org/browse/FAB-11732.

bkiran6398 commented 3 months ago

See prior discussion and workaround in Jira at https://jira.hyperledger.org/browse/FAB-11732.

Hello @denyeart , given discussion reference is not accessible. Please provide the updated link if it is persistent somewhere else.

And I believe this comment on closed PR gives a glimpse of discussion.

denyeart commented 1 month ago

from chanioxaris:

To implement this feature, changes are needed on two separate repositories:

fabric-chaincode-go (https://github.com/hyperledger/fabric-chaincode-go)|[https://github.com/hyperledger/fabric-chaincode-go]

Introduce a new API function on ChaincodeStubInterface interface, GetPrivateDataQueryResultWithPagination() that accepts a collection, query, pageSize and bookmark parameters.

fabric (https://github.com/hyperledger/fabric)|[https://github.com/hyperledger/fabric]

Changes need to be applied on chaincode handler function HandleGetQueryResult() with a new case to take into account if collection and metadata for pagination are set.

Also a new API function ExecuteQueryOnPrivateDataWithMetadata() is needed for QueryExecutor interface that accepts namespace, collection, query and metadata parameters, that returns the already implemented ExecuteQueryWithMetadata() of state db interface

Finally a new API function ExecuteQueryOnPrivateDataWithMetadata() is needed for DB interface that accepts

I have already implemented a first version of this feature, although for v1.4 and needs to be ported to v2, that seems to working as expected after comparing the chaincode results with the CouchDB direct mango query results. If anyone is interested, can have a look at these changes at this specific commit [https://github.com/tradeline-tech/fabric/commit/67e4d5842bac3bd1951bd30ed74d16a123170250] of the forked fabric repo.

denyeart commented 1 month ago

Workaround copied from Jira archive FAB-11732:

Workaround is to perform sorted queries.

For key range queries, pass the startKey of where you'd like to start or resume.
For JSON queries, utilize a sort and add a filter condition on the sorted field(s) of where you'd like to start or resume.
As you iterate through results in chaincode, it will retrieve from CouchDB in batches of 1000, as configured in core.yaml peer.ledger.state.couchDBConfig.internalQueryLimit.

hyperledger / fabric