download via windows?

[sehHeiden]

Hi,

I get a problem solely when I try to use the lib with windows 10. But not on Linux (Xubuntu 20.04)

File "Sentinel5PDownload.py", line 79, in download_sentinel5_offline sentinel5dl.download(products=result.get("products"), output_dir=_path) File "C:\Users\sebas\anaconda3\envs\Sentinel5\lib\site-packages\sentinel5dl__init.py", line 231, in download http_request(f'{base_path}/$value', filename) File "C:\Users...\anaconda3\envs\Sentinel5\lib\site-packages\sentinel5dl__init.py", line 113, in http_request os.rename(f'{filename}.tmp', filename) PermissionError: [WinError 32] Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird: 'path_to_file\S5P_OFFL_L2NO2__20190123T030718_20190123T044848_06622_01_010202_20190129T045149.nc.tmp' -> 'path_to_file\S5P_OFFL_L2NO2__20190123T030718_20190123T044848_06622_01_010202_20190129T045149.nc'

The error warning says, the process can not access the file, while it its in usage by another process.

My very unqualified guess was, that the curl connection is not yet closed, while the file already should be renamed. So added a long sleep between curl.close() and os.rename(). But this didn't help.

With kind regards

[avila-a]

Having the same problem here @Meresmata

Managed to do a very primitive patch that "solves" our main issue:

1. Go to the "C:\Users...\anaconda3\envs\Sentinel5\lib\site-packages\sentinel5dl_init.py" and do the following changes in lines 93-116

try:
        with open(f'{filename}', 'wb') if filename else io.BytesIO() as f:  # <- THIS IS CHANGED to have .nc from the beguining
            curl = pycurl.Curl()
            curl.setopt(curl.URL, url.encode('ascii', 'ignore'))
            curl.setopt(curl.USERPWD, f'{USER}:{PASS}')
            curl.setopt(curl.WRITEDATA, f)
            curl.setopt(curl.FAILONERROR, True)

            # Use a Certificate Authority (CA) bundle if set
            if ca_info:
                curl.setopt(pycurl.CAINFO, ca_info)

            # Abort if data transfer is not responding but didn't errored
            curl.setopt(pycurl.LOW_SPEED_TIME, 60)
            curl.setopt(pycurl.LOW_SPEED_LIMIT, 30)

            curl.perform()
            curl.close()

            if filename:
                # os.rename(f'{filename}.tmp', filename) <- THIS IS COMMENTED
                print("One_downloaded") # <- THIS IS INCLUDED TO AVOID ERROR
            else:
                return f.getvalue()

2. Save and restart kernel
3. Re-import the packages and run code.

Hope it works!

[lkiesow]

I have no access to a Windows machine and cannot test this but my guess would be that something (maybe cURL) on Windows uses the .tmp file to temporarily store the file during the download. Can you check if this solves the problem:

diff --git a/sentinel5dl/__init__.py b/sentinel5dl/__init__.py
index 3cdb54d..08bf8fd 100644
--- a/sentinel5dl/__init__.py
+++ b/sentinel5dl/__init__.py
@@ -91,7 +91,7 @@ def __http_request(path, filename=None, retries=9):
     url = API + path.lstrip('/')
     logger.debug('Requesting %s', url)
     try:
-        with open(f'{filename}.tmp', 'wb') if filename else io.BytesIO() as f:
+        with open(f'{filename}.temp', 'wb') if filename else io.BytesIO() as f:
             curl = pycurl.Curl()
             curl.setopt(curl.URL, url.encode('ascii', 'ignore'))
             curl.setopt(curl.USERPWD, f'{USER}:{PASS}')

[lkiesow]

No, wait, that doesn't make sense and the error is actually obvious. Will fix it in a minute…