spotify / sparkey

Simple constant key/value storage library, for read-heavy systems with infrequent large bulk inserts.
Apache License 2.0
1.18k stars 81 forks source link

File descriptor leak when hash/log files are corrupt or not matching #46

Closed seizethedave closed 2 years ago

seizethedave commented 2 years ago

In our case we had hash/log files that were not corrupt but had different file_identifier fields due to problems in a writing pipeline. The result is sparkey_hash_open() will (correctly) return -305 (SPARKEY_FILE_IDENTIFIER_MISMATCH), but the log resources will not be cleaned up, because:

The fallout is munmap and close are not called on the log file, leading to leaking file descriptors and the OS refusing to remove those bytes from the filesystem until the process is killed.

I intend to fix this problem, but figured I'd document it first!

spkrka commented 2 years ago

Thanks for the report! Since you intend to fix it, I will wait for the PR to review.