aodn / python-aodntools

Repository for templates and code relating to generating standard NetCDF files for the Australia Ocean Data Network
GNU Lesser General Public License v3.0
10 stars 3 forks source link

Aggregated Temperature Product: Permission error with temporary file #167

Closed mphemming closed 1 year ago

mphemming commented 2 years ago

File ~\\Code\python-aodntools-master\python-aodntools-master\aodntools\timeseries_products\aggregated_timeseries.py:363 in main_aggregator shutil.move(temp_outfile, os.path.join(output_dir, ncout_path))

File ~\Anaconda3\lib\shutil.py:812 in move os.unlink(src)

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Code\python-aodntools-master\python-aodntools-master\tmp9bpsj45m.nc'

I use Spyder/Jupyter notebook and I often get this error when modifying a NetCDF file while a file of the same name is currently imported into the workspace.

The LTSP is created successfully, but the temporary file is left unmodified.

mphemming commented 1 year ago

So the issue is that the temp file is still open in Spyder, meaning that shutils cannot move the file at the end of the main aggregator function.

I edited line 214 to include 'fd' as output: _, temp_outfile = tempfile.mkstemp(suffix='.nc', dir=output_dir) to fd, temp_outfile = tempfile.mkstemp(suffix='.nc', dir=output_dir).

I can then close the object adding the following code to line 355 prior to using 'shutil.move':

os.close(fd)

This now works without an error for both 'aggregated_timeseries.py' and 'velocity_aggregated_timeseries.py'. Perhaps the AODN can consider editing the code to include this.

mhidas commented 1 year ago

Thanks @mphemming The most efficient way to propose changes to the code is to fork the repository, commit your changes, and submit a Pull Request. Then we can review it and merge it in once finalised. Does that work for you?