kvStore_Purge is not removing keys

[EgusCpp]

Observed behavior

I am creating a bucket for key-value pairs, using FileStorage for persistence. Once I fill storage with values using kvStore_CreateString, I am able to verify keys and values with kvStore_Get, and all of them are good. However, if I want to delete all using kvStore_Purge, keys and values are still sitting in the storage, despite kvStore_Purge returns NATS_OK. Calling any of kvStore_Status and kvStore_Keys I can check number of records remaining the same as I put there before deletion.

Expected behavior

According to documentation, kvStore_Purge is expected to delete key and all revisions. As I am not using revisions, I would expect kvStore_Delete to do the same.

Server and client version

NATS windows server v.2.10.6 nats.c client library 3.7.1-beta

Host environment

OS:

Edition Windows 11 Pro
Version 21H2
Installed on    ‎2022-‎02-‎18
OS build    22000.1936
Experience  Windows Feature Experience Pack 1000.22001.1000.0

Hardware:

Device name DESKTOP-NOH9FND
Processor   11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz   2.50 GHz
Installed RAM   64.0 GB (63.7 GB usable)

Steps to reproduce

1) create bucket

kvConfig_Init(&kv_cfg);
kv_cfg.Bucket = "orders";
kv_cfg.StorageType = js_FileStorage;

auto s = js_KeyValue(&kv, js, "orders");
if (s != NATS_OK) {
    s = js_CreateKeyValue(&kv, js, &kv_cfg);
    if (s != NATS_OK) {
        std::cout << "js_CreateKeyValue failed with " << s << std::endl;
        exit(0);
    }
}

2) put KV to storage

for (unsigned int idx = 0; idx < SAMPLES_CNT; idx++) {
    key = std::to_string(idx);
    value = __getRandomString(100, idx);
    storedValues[key] = value;
    if (kvStore_CreateString(NULL, kv, key.c_str(), value.c_str()) != NATS_OK) {
        step1_write_errors++;
    }
}

3) erase keys

for (const auto& k_it : storedValues) {
    s = kvStore_Purge(kv, k_it.first.c_str(), NULL);
    if (s != NATS_OK) {
        std::cout << "kvStore_Purge failed with " << s << std::endl;
        step3_erase_errors++;
    }
}

4) check keys number

kvKeysList kList;
s = kvStore_Keys(&kList, kv, NULL);
if (s == NATS_OK) {
    std::cout << "kvStore has " << kList.Count << " keys" << std::endl;
}
kvKeysList_Destroy(&kList);

NOTE: if I pass options to kvStore_Purge filled as

kvPurgeOptions kop;
kvPurgeOptions_Init(&kop);
kop.DeleteMarkersOlderThan = 0;
kop.Timeout = 0;

then it changes nothing, i.e. records are still not removed

[EgusCpp]

Update: nats CLI tool showing there is no keys (using "kv ls bucket_name"), while "kv info" shows there are still values in this bucket. Maybe it's a bug in the server itself?

[ripienaar]

Purge replace all the values for a key with a single delete marker, the key will remain but marked as deleted. You can later delete purged keys with nats kv compact

[EgusCpp]

@ripienaar Thank you, seems I overlooked this function, as it seems to be missing for client library.

[ripienaar]

I believe only the go client has the compact function but might be wrong.

[EgusCpp]

Hello @kozlovic ! Could we expect "kv compact" to be added to C client library in the near future?

[kozlovic]

I believe that this is what kvStore_PurgeDeletes does (http://nats-io.github.io/nats.c/group__kv_group.html#ga34ab74b229ff88e64301ea53a274da69)

[EgusCpp]

Yes, that's it, thank you! Got confused by naming, while description is not obvious when you didn't get familiar with internal structure yet.