to_raster 'compress' functionality 'FileExistsError:'

[SebastianLeh]

I am exporting a raster with compress='lzw' option. As I understand, 'to_raster' saves the file, then compresses it and renames it to the original filename. I get a windows error message stating that the file already exists, which makes sense to me. Is there something I am missing? Is there an argument to pass a new filename for the compressed raster or the like? I am using geowombat version 2.1.0. Thanks for your help.

Console output as follows: Input In [2] in <cell line: 1> export_raster.gw.to_raster(results,overwrite=True, bigtiff=True,nodata=nodata,n_jobs=8, verbose=1, compress='lzw')

File ~\miniconda3\envs\data_hand3.9\lib\site-packages\geowombat\core\geoxarray.py:789 in to_raster to_raster(

File ~\miniconda3\envs\data_hand3.9\lib\site-packages\geowombat\core\io.py:1270 in to_raster temp_file.rename(filename)

File ~\miniconda3\envs\data_hand3.9\lib\pathlib.py:1382 in rename self._accessor.rename(self, target)

FileExistsError: [WinError 183] Cannot create a file when that file already exists: [path to the file or folder in question]

[jgrss]

Hi @SebastianLeh,

Is it possible that there is a lock on the file? I cannot reproduce the error if I run the code snippet below multiple times.

import geowombat as gw
from geowombat.data import l8_224078_20200518_B2

with gw.open(l8_224078_20200518_B2) as src:
    src.gw.to_raster(
        'test.tif', overwrite=True, bigtiff=True, nodata=0, n_jobs=2, verbose=1, compress='lzw', overwrite=True
    )

Could you test this ☝️ on your system and let me know if that works for you?

[SebastianLeh]

Thanks for your reply!

I ran the code on my system and a second system and in both cases I still get 'FileExistsError: [WinError 183]'.

It appears to be the correct behaviour of os.rename on Windows according to this post: https://stackoverflow.com/questions/8107352/force-overwrite-in-os-rename

When I exchange 'os.rename' with 'os.replace' (in line 1270 of io.py), as suggested in the referenced post, it works and the existing file is replaced with the compressed file.

[jgrss]

Thanks for raising this. This appears to be limited to Windows, which is likely why I couldn't replicate the error. We use pathlib, which confirms what you found:

Path.rename(target)

Rename this file or directory to the given target, and return a new Path instance pointing to target. 
On Unix, if target exists and is a file, it will be replaced silently if the user has permission. On Windows, 
if target exists, [FileExistsError](https://docs.python.org/3/library/exceptions.html#FileExistsError) will 
be raised. target can be either a string or another path object:

I will create a PR to change temp_file.rename(filename) to temp_file.replace(filename)

[jgrss]

@SebastianLeh I am going to close this with #244, but please re-open if the problem persists.