How To Achieving Maximum Single-Core Read/Write QPS

[xiaguan]

I've implemented a nvme-sdd hash storage engine using Seastar. Its core is a Worker class, which maintains a memory hash table and metadata internally. It's not thread-safe and can only run on a single thread.

Currently, I have implemented basic put and get interfaces, and wrapped them with KVService:

C++
class KVService {
public:
  KVService(Worker worker) : _worker(worker) {}

  seastar::future<bool> put(const std::string &key, const std::string &value) {
    co_return co_await _worker.put(key, value);
  }

  seastar::future<std::optional<std::string>> get(const std::string &key) {
    co_return co_await _worker.get(key);
  }

private:
  Worker _worker;
};

// for simple
class Worker {
private:
    unorder_map<key,(page_num,offset)> _map;
};

After simple verification, the functionality is working correctly. However, I encountered difficulties when I wanted to benchmark it, i.e., starting multiple clients for concurrent testing.

Problems:

• How to implement multiple clients concurrently benchmarking KVService in Seastar?
• Looking for any design and implementation advice for a single-core NVMe SSD hash engine

[xiaguan]

My idea is to leverage the shared-nothing nature of both the hash engine and Seastar itself. This could potentially lead to the creation of a KV engine with impressive point read and write performance, specifically tailored for workloads that don't require range scans.

[tchaikov]

.

Problems:

* How to implement multiple clients concurrently benchmarking KVService in Seastar?

* Looking for any design and implementation advice for a single-core NVMe SSD hash engine

IIUC, these are topics for discussion not seastar issues, please use https://github.com/scylladb/seastar/discussions for discussions. i am closing this issue for now.