file descriptor leak on `index.build`

[drbh]

calling index.build successively seems to cause a file descriptor leak.

When calling insert and build many times (~120) some file is not closed and this causes ngt to crash. Below is the output of a small test that inserts and builds.

inserted vector with id 107
inserted vector with id 108
inserted vector with id 109
inserted vector with id 110
inserted vector with id 111
inserted vector with id 112
inserted vector with id 113
Error: Error("Capi : ngt_save_index() : Error: /Users/drbh/Projects/ngt-rs/ngt-sys/NGT/lib/NGT/ObjectRepository.h:47: NGT::ObjectSpace: Cannot open the specified file /var/folders/0n/s24tgvhd60xghdtn1wz4z5wm0000gn/T/.tmp87SDPW/obj.")

---- index::tests::test_multithreaded stdout ----
Error: Custom { kind: Uncategorized, error: PathError { path: "/var/folders/0n/s24tgvhd60xghdtn1wz4z5wm0000gn/T/.tmpdwhCs3", err: Os { code: 24, kind: Uncategorized, message: "Too many open files" } } }

failures:
    index::tests::test_incremental_insert_and_build
    index::tests::test_multithreaded

test to reproduce this leak is available here: https://github.com/lerouxrgd/ngt-rs/pull/12

TLDR the test;

for _ in 0..120 {
    let vec = vec![1.0, 2.0, 3.0];
    let id = index.insert(vec.clone())?;
    println!("inserted vector with id {}", id);

    // Build and persist the index
    index.build(1)?; // <------------------- LEAKS HERE
    index.persist()?;
}

I believe this is a file descriptor issue because

1. this causes the multithread test to fail with "Too many open files"
2. running lsof on a binary that calls insert and build shows a growing number of /dev/null and /dev/ttys006 files

I'm not exactly sure where this issue originates, and it does not appear that other ngt bindings explicitly close files after calling build (create_index), however I may be missing something simple that will close these files and avoid the crash.

Please let me know if I can provide any more information!

[lerouxrgd]

Hello, I saw that you reported the underlying issue to NGT, I will update ngt-rs accordingly very soon. (I am doing some refactoring for NGT 2 and its quantized indexes)

[drbh]

amazing! thank you for the great project 🙏