Intermittent stream_path error on downloading some files

Hi HDX Team!

I am carrying out analysis on some datasets from Kenya, part of which requires I download tabular data. This works really well most of the time, but in calling ...

dataset = Dataset.read_from_hdx(row["id"])
resources = dataset.get_resources()
for resource in resources:
     url, path = resource.download(dir)

We sometimes get ...

Traceback (most recent call last): File "", line 744, in download_data url, path = resource.download(dir) File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dbff6b46-9075-4f38-a1a6-07585d60da84/lib/python3.9/site-packages/hdx/data/resource.py", line 515, in download path = downloader.download_file( File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dbff6b46-9075-4f38-a1a6-07585d60da84/lib/python3.9/site-packages/hdx/utilities/downloader.py", line 428, in download_file return self.stream_path( File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dbff6b46-9075-4f38-a1a6-07585d60da84/lib/python3.9/site-packages/hdx/utilities/downloader.py", line 347, in stream_path raise DownloadError(errormsg) from e hdx.utilities.base_downloader.DownloadError: Download of https://data.humdata.org/dataset/db3e1a76-76d8-4206-9e5e-382336c51472/resource/562aad8a-2d68-4560-8c8b-237459958183/download/dtm_kenya_b2_baseleine_multi_sectoral_assessment_nov_2022.xlsx failed in retrieval of stream!

However, if I use the URL in a browser ...

https://data.humdata.org/dataset/db3e1a76-76d8-4206-9e5e-382336c51472/resource/562aad8a-2d68-4560-8c8b-237459958183/download/dtm_kenya_b2_baseleine_multi_sectoral_assessment_nov_2022.xlsx

It seems to work fine and I get a file.

I am using hdx-python-api==6.0.8 and Python 3.9.5.

Thanks a lot!

OCHA-DAP / hdx-python-api

Intermittent stream_path error on downloading some files #48