Closed flowers9 closed 7 years ago
We will eventually change the tracking mechanism to use code developed recently by @pb-cdunn which can solve this problem if the latency of the file system synchronization through the different nodes is not too bad.
In unzip.py, wait_for_file(), the check for a "done" file can fail even if the done file is present if the remote file system updates the presence of the "exit" file before the "done" file (this is probably more likely when the remote file system is heavily loaded). Introducing a simple delay in the function can prevent a lot of extraneous restarting.
(On the plus side, all the restarting led me to spot the bug I posted about earlier. ;)