search

peterbourgon / diskv

A disk-backed key-value store.
http://godoc.org/github.com/peterbourgon/diskv
MIT License
1.41k stars 102 forks source link

Implement a fix for #66, excessive memory use in siphon. #67

Open floren opened 3 years ago

floren commented 3 years ago

The siphon will now stop writing to its internal buffer once the size of the buffer exceeds the maximum cache size. Because we write until we exceed the max cache size, we're safe to attempt the cache update even if the buffer only contains partial data, because it's still over the limit & will be rejected.

peterbourgon commented 3 years ago

Would it be smarter to check both d.CacheSizeMax and fi.Size in readWithRLock, and only create and use the siphon if the file size could conceivably fit in the cache?