fani-lab / Adila

Fairness-Aware Team Formation
3 stars 2 forks source link

Multiprocess access to the same file or folder #44

Open hosseinfani opened 1 year ago

hosseinfani commented 1 year ago

@Hamedloghmani When running Adila in parallel for multiple *.pred files, we rely on file locking system of os. However, there will be some issues like

One process may be true in condition check, but when creating see the folder exists due to other process! https://github.com/fani-lab/Adila/blob/040b34b105203156bd0a08b4789006c8dabc0631/src/main.py#L170

Or when logging the time at: https://github.com/fani-lab/Adila/blob/040b34b105203156bd0a08b4789006c8dabc0631/src/main.py#L197

Temporary solution is to run sequentially for one pred file. Then, run in parallel for all files.

Hamedloghmani commented 1 year ago

@hosseinfani Thank you for pointing that out. That was a really solid point. I think this was the reason that on unix processes became zombies and running would not finish in some cases, although Windows handled that. I will look for a permanent solution.

hosseinfani commented 1 year ago

@Hamedloghmani put it in the lowest priority, pls. Not important at all for now