search

microsoft / garnet

Garnet is a remote cache-store from Microsoft Research that offers strong performance (throughput and latency), scalability, storage, recovery, cluster sharding, key migration, and replication features. Garnet can work with existing Redis clients.
https://microsoft.github.io/garnet/
MIT License
10.13k stars 499 forks source link

Investigate use of RandomAccess API to implement storage device in Tsavorite #89

Open PaulusParssinen opened 5 months ago

PaulusParssinen commented 5 months ago

We discussed this a bit earlier in the Garnet discord but I'm opening issue for tracking it here.

In .NET 6, the File IO in the BCL was essentially entirely rewritten, including introduction of new RandomAccess low-level API. The RandomAccess API is extremely optimized low-level offset based thread-safe IO wrapper that depending on the platform, either uses pread(v)/pwrite(v) for Unix or Overlapped IO for Windows.

We should investigate if RandomAccess based storage device implementation can be added to Tsavorite.core.

AlgorithmsAreCool commented 5 months ago

I've personally had great success with RandomAccess, but I'm curious about the perceived advantages of this addition? Performance? Consolidation?