support querying entries under a vstorage key in one abci_query request

[turadg]

What is the Problem Being Solved?

Querying a subtree of a vstorage key requires querying each node separately. The network request can be optimized with HTTP batching in CosmJS but the RPC server would still do N abci_query.

This isn't a problem for one level deep because multiple abci_query in the same HTTP request is cheap. Getting the data of all children is just two HTTP queries using the gRPC protobuf.

example from @gibson042

$ path_prefix=highPrioritySenders; \
  child_segments="$(\
    curl -sSL \
      'http://devnet.rpc.agoric.net/' \
      -X POST \
      -H 'Content-Type: application/json' \
      --data "$(\
        jq -n \
          --arg prefix "$path_prefix" \
          '{
            jsonrpc: "2.0",
            id: 1,
            method: "abci_query",
            params: { path: ("custom/vstorage/children/" + $prefix) },
          }' \
      )" | \
    jq '.result.response.value | @base64d | fromjson | .children' \
  )"; \
  curl -sSL \
    'http://devnet.rpc.agoric.net/' \
    -X POST \
    -H 'Content-Type: application/json' \
    --data "$(\
      jq -n \
        --arg prefix "$path_prefix" \
        --argjson arr "$child_segments" \
        '$arr | to_entries | map({
          jsonrpc: "2.0",
          id: (.key + 1),
          method: "abci_query",
          params: { path: ("custom/vstorage/data/" + $prefix + "." + .value) },
        })'
    )" | \
    jq \
      --argjson child_segments "$child_segments" \
      '
        map({
          key: $child_segments[.id - 1],
          value: .result.response.value | @base64d,
        }) |
        from_entries
      '

{
  "agoric140dmkrz2e42ergjj7gyvejhzmjzurvqeq82ang": "{\n  \"value\": \"economicCommittee\"\n}",
  "agoric1cuwd8uhemfx26fmzvrdy295ltscyl350mz5kek": "{\n  \"value\": \"ATOM-USD_price_feed\"\n}",
  "agoric1ldmtatp24qlllgxmrsjzcpe20fvlkp448zcuce": "{\n  \"value\": \"economicCommittee\"\n}"
}

However, fetching a second level deep will require as many HTTP requests as there are children of the parent.

Also some servers provide only the REST interface, which doesn't support batching even at one level.

Description of the Design

Make vstorage allow for a single abci_query for a whole subtree. It would use the /entries value that vstorage has now.

For this issue to be complete, support has to be wired through,

• vstorage exposing to abci_query
• Casting supporting querying of /entries format
• clients consuming entries

Smart wallet publishing will also need to be updated to take advantage of that, but it can be tackled separately once this issue is resolved.

Security Considerations

Scaling Considerations

Should reduce writeops and not increase read ops.

Test Plan

Clients can queries an entry set and be efficiently notified of changes.

[mhofman]

Wondering if we should immediately bake in concepts such as pagination, and entries order. If the client uses lexically increasing offerIds, it could make a query to receive the entries in reverse order, as I assume more recent offers are likely more relevant to display.

However I don't know how easily we can plumb reverse iteration and "after key" for pagination in vstorage. The capability does exist in the underlying kvStore iterators afaik.