Open sudheesh-sas opened 2 months ago
From this.
`def download( url, outputfile, writeType): global responseText r = requests.get(url)
#r.status_code=409
#responseText=r.text
r.raise_for_status()
with open(outputfile, writeType) as f:
f.write(r.content) # write data to file`
Modify to:
`def download( url, outputfile, writeType): global responseText r = requests.get(url)
#r.status_code=409
#responseText=r.text
r.raise_for_status()
#with open(outputfile, writeType) as f:
# f.write(r.content) # write data to file
# Retry for PermissionError in the open statement
retry_attempts = 5
for attempt in range(retry_attempts):
try:
with open(outputfile, writeType) as f:
f.write(r.content) # Write data to file
break # If the operation succeeds, exit the loop
except PermissionError:
if attempt < retry_attempts - 1: # Don't sleep after the last attempt
print(f"PermissionError occurred. Retry {attempt + 1}/{retry_attempts} after 1 second.")
time.sleep(2) # Wait for 1 second before retrying
else:
raise # If it fails after the max retries, raise the exception`
While downloading files, there is an issue where the same file is being accessed by more than one process. Line # 394 for discover.py download( url, outputfile, writeType)