search

CoreyKaylor / Lightning.NET

.NET library for LMDB key-value store
Other
398 stars 82 forks source link

Stream interface #83

Closed lbergnehr closed 4 years ago

lbergnehr commented 7 years ago

In order to avoid keeping large chunks of data in memory at the same time as well as making it easier to store really large objects, it would be nice to provide a Stream interface to reading and putting data to the database.

Moreover, instead of copying the data with Marshal.Copy, would it be possible to use something like UnmanagedMemoryStream instead, avoiding copying the data at all?

CoreyKaylor commented 7 years ago

I like the proposal on this issue. https://github.com/CoreyKaylor/Lightning.NET/issues/85

Regarding the UnmanagedMemoryStream I saw worse performance using it over the straight Marshal.Copy. That doesn't mean that a combination of the two possibly wouldn't get us to a better place. Again, happy to take a PR for this especially if it can demonstrably gain perf benefits.

CoreyKaylor commented 7 years ago

https://github.com/dotnet/corefx/issues/18946

This API might provide what we need to achieve something of this nature.

CoreyKaylor commented 4 years ago

Closing in favor of https://github.com/CoreyKaylor/Lightning.NET/issues/110