epfml / sent2vec

General purpose unsupervised sentence representations
Other
1.19k stars 256 forks source link

Fix: Safeguard access to shared memory during matrix loading #59

Closed ivonindza closed 5 years ago

ivonindza commented 5 years ago

Processes started concurrently could access shared memory before the input matrix was loaded into it. Now, the matrix is loaded into a different path than the one processes expect and hardlinked with the proper one once the loading is finished. Processes may have to wait at startup for this to finish, and an optional timeout parameter is added to prevent them from waiting forever in case of error.