Closed bgunnar5 closed 4 months ago
Can you briefly explain the race condition and how this would fix it?
In step.py
where you last commented, the status was being dumped using:
with open(status_filepath, "w") as status_file:
json.dump(status_info, status_file)
In this case, the status is being dumped to the status file and not using a file lock object to ensure nothing else is reading/writing to it at the same time. I changed this to call:
write_status(status_info, status_filepath, f"{self.workspace.value}/status.lock")
With this change, it ensures that a file lock is being used every time the status file is being written to here.
@lucpeterson just added two more things to this:
merlin status
from v1.12.1 on a study that was ran in v1.12.0.Once I get the merge conflicts resolved this should be good to go
This branch should fix the problems discussed in my first comment of #469. However, I was never able to reproduce the initial bug that was posted about in that issue thread.
This branch should also allow for safer handling of the
JSONDecodeError
that users were seeing as I added a try/except for it. I think this error may have been arising due to a race condition that I failed to recognize initially. I tried to safeguard around this race condition by creating thewrite_status
function. Hopefully this makes it so theJSONDecodeError
is gone now.