k3s-io / kine

Run Kubernetes on MySQL, Postgres, sqlite, dqlite, not etcd.
Apache License 2.0
1.49k stars 226 forks source link

fixes the incorrect total items count in a paginated list items q… #271

Closed HemantNegi closed 5 months ago

HemantNegi commented 5 months ago

When making a list query the returned total matched items count is wrong.

Steps to replicate: create some keys in the database with specific prefix say /registry then query with limit clause.

kine ❯ etcdctl get --prefix /registry --limit 1 --write-out=json | jq
{
  "header": {
    "revision": 494943
  },
  "kvs": [
    {
      "key": "L3JlZ2lzdHJ5L2hlYWx0aA==",
      "create_revision": 2,
      "mod_revision": 2,
      "value": "eyJoZWFsdGgiOiJ0cnVlIn0="
    }
  ],
  "more": true,
  "count": 2
}

the count shows wrong value.

After this Fix

kine ❯ etcdctl get --prefix /registry --limit 1 --write-out=json | jq
{
  "header": {
    "revision": 495036
  },
  "kvs": [
    {
      "key": "L3JlZ2lzdHJ5L2hlYWx0aA==",
      "create_revision": 2,
      "mod_revision": 2,
      "value": "eyJoZWFsdGgiOiJ0cnVlIn0="
    }
  ],
  "more": true,
  "count": 1842
}
HemantNegi commented 5 months ago

This problem was originally noticed while we were using Kine with a k8s api server. We make a paginated list objects api call and it returned incorrect remainingItemCount.

Please consider merging this or else fix this issue.

brandond commented 5 months ago

Can also test this with:

kubectl proxy &
curl -s 'http://localhost:8001/api/v1/namespaces/kube-system/pods?limit=1&resourceVersion=700&resourceVersionMatch=Exact'

On a default k3s cluster shortly after startup, there should be 6 items remaining.

HemantNegi commented 5 months ago

Thankyou for taking quick action on this and fixing other case as well. Looking forward to get it merged.