Race Condition on Mac OS with file system locking

elast0ny / shared_memory

A Rust wrapper around native shared memory for Linux and Windows

380 stars 51 forks source link

Race Condition on Mac OS with file system locking #71

Closed elast0ny closed 3 years ago

elast0ny commented 3 years ago

Unfortunately, the dependency i chose for file system locks seems to be broken on MacOS.

The race happens when process 1 creates a file that is supposed to contain the shared memory mapping unique ID but process 2 ends up reading the file before the ID is written by process 1...

I will have to either find a new file locking crate or be OK with not fully supporting MacOS for now.

p-avital commented 3 years ago

I'll try investigating this today for a bit. Do you have any reproduction guide? Also, given that this bug, as far as I know, is present on 0.11, could you still publish a new version? That would really help on my project's end.

p-avital commented 3 years ago

After investing a bit on my mac :

All of fs2's tests are passing, so even if their tests were running in their CI, they don't seem to be able to catch this case.
The issue with opening the file isn't systematic, so it's going to be hard to snuff out, but running the example repeatedly does get it to fail once every 5 to 10 runs. I'm not sure I can do much to fix the core issue, but since the race can be detected, maybe the MacOS version should have a version of open that backs off when detecting the race, and retries a few times ? That I should be able to do a PR for, let me know if that'd suit you :)

elast0ny commented 3 years ago

Oh yeah great idea. If I understand correctly, you're saying we could add a number of retries when reading a shmem ID from a flink ? Maybe loop up to 5 times with a slight delay trying to read the shared memory ID from the file to give process 1 some time to write it completely ?

We could even reduce the number of dependencies for shared_memory and remove fs2 completely so all OS implementations would rely on this retry logic.

p-avital commented 3 years ago

I'm honestly not familiar enough with fs2 to be able to tell if removing it would be relevant for the platforms where it works properly.

If that's OK with you, I'll start working on the retry mechanism for mac. once it's done, extending it to linux and benching it against the current version would probably be for the best :)

elast0ny commented 3 years ago

Fixed in #74