curiosity-ai / rocksdb-sharp

.net bindings for the rocksdb by facebook
BSD 2-Clause "Simplified" License
155 stars 38 forks source link

DB created with RocksDb.OpenReadOnly not releasing file handles after disposing + Directory.Delete #10

Closed dinardov closed 3 years ago

dinardov commented 3 years ago

I'm currently running into an issue where the files a RocksDb instance used to point to still exist after disposing and doing a Directory.Delete(<path>) on those files. Here's the sequence so far:

  1. I create a RocksDb instance with RocksDb.OpenReadOnly(<options>, <path>, true).
  2. I want to swap it with a new RocksDb instance (also created with RocksDb.OpenReadOnly) I have staged with all new data (at a different path/directory).
  3. After I point the current instance to the new instance (keeping track of the old instance), I call Dispose() on the old instance and call Directory.Delete(<path>).
  4. Running lsof (list of open files) while the process is still running, I see that the *.sst files are marked as (deleted), but they still have file handles open; even though the files and directory don't exist, they take up disk space, even after many hours, seemingly indefinitely (this swapping part from 2 and 3 repeat every few hours, so eventually disk space fills up).

Also tried to call Native.Instance.rocksdb_destroy_db(dboptions.Handle, <directory-name>); which disposes and deletes the directory, but it still doesn't release the file handles.

EDIT: Also tried replacing all of the RocksDb.OpenReadOnly calls with RocksDb.Open and the same issue persists.

dinardov commented 3 years ago

After careful re-examination of code using Iterators, there was an edge case where an exception was being thrown and skipping disposal prior to calling Dispose() on the Iterator. I would recommend always using a using block for iterators to prevent this, and if that's not possibly doing a try/catch/finally block immediately after Iterator declaration, and calling Dispose() on the Iterator in finally.

Thanks to this comment for the hint to reassess Iterators!

Also cross-referencing this question with duplicate on facebook/rocksdb repo.