Open arteymix opened 1 week ago
This is going to be important for single-cell data since those file take significantly longer to produce.
I've added a read-write lock to wait when a file is being generated and prevent multiple threads from generating the same file.
The next step is to support file-level locking. This is less of an issue though because most of the volume will occur on the Web server.
I did experiment a little bit with Java FileChannel API which allows a Java process to acquire an exclusive lock on a file (not but a directory, which is problematic for MEX).
In addition, within Gemma, we should have a read-write lock for reading the contents of the file. The FileLock object is not meant for protecting a file within the JVM context.
There might be implication of doing that with NFS.
Until this is implemented, users are subject to receive potentially incomplete/corrupted files.