search

tahoe-lafs / zfec

zfec -- an efficient, portable erasure coding tool
Other
373 stars 44 forks source link

Require explicit, single-threaded initialization #91

Closed exarkun closed 9 months ago

exarkun commented 9 months ago

Fixes #89

This changes the underlying C library to require an initial call to fec_init before fec_new (and therefore the other APIs) can succeed. This avoids the problem where multiple threads call fec_new and implicitly trigger parallel initialization leading to corruption of some zfec internal state.

Calls to the C function fec_new will now return NULL if a call to the C function fec_init has not yet completed.

This change is not visible to the Python bindings because (a) they always call into fec while holding the GIL and so there is no chance for multithreaded usage and (b) the Python module initializer in the bindings now calls the initialization function automatically.

This change is visible to the Haskell bindings because the Haskell API can really make parallel calls into the fec library from multiple OS threads. Therefore, there is a new Haskell API, Codec.FEC.initialize, which must be used before Codec.FEC.new. If it is not, Codec.FEC.new will throw an exception.

This is an incompatible change to the Haskell API so it comes with a version bump for those bindings.

exarkun commented 9 months ago

There was a segfault during one of the macOS test runs. I managed to reproduce this (at least, it appears to show up in the same test method) on my Linux laptop on the branch (after ~800 iterations) and on master (after ~1800 iterations) - so I'm supposing for now that's some pre-existing issue.

itamarst commented 9 months ago

Might be able to find the problem with one of clang's sanitizers, or with valgrind.