This is a minor bug, but odd none the less. I noted that files (specifically hdf5 format) that were created with _do_chunk_output (or equivalent) can be opened, read, and closed programmatically with h5py, but when I used a GUI plug in for my IDE, the files were considered invalid unless I opened then, importantly, closed them with h5py.
Additionally, when the files were written in a single shot, without ever producing a temporary, "inprogress_.hdf5" file, they were correctly formatted, and presumably closed by some mechanism inside RAIL.
After some investigation, I found RailStage._finalize_tag contained conditional logic that would call handle.write() (which will close a file) only if the file handle didn't already exist. Removing the conditional check for file existence, but keeping handle.write() seems to fix the problem and luckily doesn't obviously introduce collateral problems.
The unit tests all pass, and my notebooks run, but I don't have high confidence that there are not other problems introduced by the change.
Before submitting
Please check the following:
[ ] I have described the situation in which the bug arose, including what code was executed, information about my environment, and any applicable data others will need to reproduce the problem.
[ ] I have included available evidence of the unexpected behavior (including error messages, screenshots, and/or plots) as well as a descriprion of what I expected instead.
[ ] If I have a solution in mind, I have provided an explanation and/or pseudocode and/or task list.
This is a minor bug, but odd none the less. I noted that files (specifically hdf5 format) that were created with
_do_chunk_output(or equivalent) can be opened, read, and closed programmatically with h5py, but when I used a GUI plug in for my IDE, the files were considered invalid unless I opened then, importantly, closed them with h5py.Additionally, when the files were written in a single shot, without ever producing a temporary, "inprogress_.hdf5" file, they were correctly formatted, and presumably closed by some mechanism inside RAIL.
After some investigation, I found
RailStage._finalize_tagcontained conditional logic that would callhandle.write()(which will close a file) only if the file handle didn't already exist. Removing the conditional check for file existence, but keepinghandle.write()seems to fix the problem and luckily doesn't obviously introduce collateral problems.The unit tests all pass, and my notebooks run, but I don't have high confidence that there are not other problems introduced by the change.
Before submitting Please check the following: