Errors when running all datasets except the ESRI World Imagery

[michaeltrent]

I've been using this module for serval months with no issues. Suddenly two weeks ago I started getting the following errors when trying to predict anywhere in the US and at any inference window size:

During handling of the above exception, another exception occurred:

[2023-06-22 11:02:04,688] - Touching session (e55bc7b7c3c84c32bb43881ab21bacec) Found 2 intersections, returning at 1_3_2019/data/v1/2012/states/tx/tx_1m_2012/31097/m_3109731_se_14_1_20120728_probs.tif Found 2 intersections, returning at 1_3_2019/data/v1/2012/states/tx/tx_1m_2012/31097/m_3109731_se_14_1_20120728_probs.tif Traceback (most recent call last): File "rasterio/_base.pyx", line 216, in rasterio._base.DatasetBase.init File "rasterio/_shim.pyx", line 67, in rasterio._shim.open_dataset File "rasterio/_err.pyx", line 205, in rasterio._err.exc_wrap_pointer rasterio._err.CPLE_HttpResponseError: HTTP response code: 409

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/bottle.py", line 868, in _handle return route.call(args) File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/bottle.py", line 1748, in wrapper rv = callback(*a, *ka) File "server.py", line 376, in get_input input_raster = current_data_loader.get_data_from_extent(extent) File "/home/mtrent/landcover/web_tool/DataLoader.py", line 555, in get_data_from_extent with rasterio.open(LC_BLOB_ROOT + "/" + naip_fn) as f: File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/rasterio/env.py", line 445, in wrapper return f(args, kwds) File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/rasterio/init.py", line 219, in open s = DatasetReader(path, driver=driver, sharing=sharing, **kwargs) File "rasterio/_base.pyx", line 218, in rasterio._base.DatasetBase.init rasterio.errors.RasterioIOError: HTTP response code: 409 Traceback (most recent call last): File "rasterio/_base.pyx", line 216, in rasterio._base.DatasetBase.init File "rasterio/_shim.pyx", line 67, in rasterio._shim.open_dataset File "rasterio/_err.pyx", line 205, in rasterio._err.exc_wrap_pointer rasterio._err.CPLE_HttpResponseError: HTTP response code: 409

Traceback (most recent call last): File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/bottle.py", line 868, in _handle return route.call(args) File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/bottle.py", line 1748, in wrapper rv = callback(*a, *ka) File "server.py", line 229, in pred_patch input_raster = current_data_loader.get_data_from_extent(extent) File "/home/mtrent/landcover/web_tool/DataLoader.py", line 555, in get_data_from_extent with rasterio.open(LC_BLOB_ROOT + "/" + naip_fn) as f: File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/rasterio/env.py", line 445, in wrapper return f(args, kwds) File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/rasterio/init.py", line 219, in open s = DatasetReader(path, driver=driver, sharing=sharing, **kwargs) File "rasterio/_base.pyx", line 218, in rasterio._base.DatasetBase.init rasterio.errors.RasterioIOError: HTTP response code: 409

I also occasionally get this error:

[2023-06-22 11:05:56,774] - Touching session (981d5f491f9c4f8ba80172eff7874f67) Found 1 intersections, returning at 1_3_2019/data/v1/2012/states/tx/tx_1m_2012/31097/m_3109762_nw_14_1_20120728_probs.tif Found 1 intersections, returning at 1_3_2019/data/v1/2012/states/tx/tx_1m_2012/31097/m_3109762_nw_14_1_20120728_probs.tif Traceback (most recent call last): File "rasterio/_base.pyx", line 216, in rasterio._base.DatasetBase.init File "rasterio/_shim.pyx", line 67, in rasterio._shim.open_dataset File "rasterio/_err.pyx", line 205, in rasterio._err.exc_wrap_pointer rasterio._err.CPLE_OpenFailedError: '/vsicurl/https://modeloutput.blob.core.windows.net/full-usa-output/1_3_2019/data/v1/2012/states/tx/tx_1m_2012/31097/m_3109762_nw_14_1_20120728_probs.tif' does not exist in the file system, and is not recognized as a supported dataset name.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/bottle.py", line 868, in _handle return route.call(args) File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/bottle.py", line 1748, in wrapper rv = callback(*a, *ka) File "server.py", line 229, in pred_patch input_raster = current_data_loader.get_data_from_extent(extent) File "/home/mtrent/landcover/web_tool/DataLoader.py", line 555, in get_data_from_extent with rasterio.open(LC_BLOB_ROOT + "/" + naip_fn) as f: File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/rasterio/env.py", line 445, in wrapper return f(args, kwds) File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/rasterio/init.py", line 219, in open s = DatasetReader(path, driver=driver, sharing=sharing, **kwargs) File "rasterio/_base.pyx", line 218, in rasterio._base.DatasetBase.init rasterio.errors.RasterioIOError: '/vsicurl/https://modeloutput.blob.core.windows.net/full-usa-output/1_3_2019/data/v1/2012/states/tx/tx_1m_2012/31097/m_3109762_nw_14_1_20120728_probs.tif' does not exist in the file system, and is not recognized as a supported dataset name. Traceback (most recent call last): File "rasterio/_base.pyx", line 216, in rasterio._base.DatasetBase.init File "rasterio/_shim.pyx", line 67, in rasterio._shim.open_dataset File "rasterio/_err.pyx", line 205, in rasterio._err.exc_wrap_pointer rasterio._err.CPLE_OpenFailedError: '/vsicurl/https://modeloutput.blob.core.windows.net/full-usa-output/1_3_2019/data/v1/2012/states/tx/tx_1m_2012/31097/m_3109762_nw_14_1_20120728_probs.tif' does not exist in the file system, and is not recognized as a supported dataset name.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/bottle.py", line 868, in _handle return route.call(args) File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/bottle.py", line 1748, in wrapper rv = callback(*a, *ka) File "server.py", line 376, in get_input input_raster = current_data_loader.get_data_from_extent(extent) File "/home/mtrent/landcover/web_tool/DataLoader.py", line 555, in get_data_from_extent with rasterio.open(LC_BLOB_ROOT + "/" + naip_fn) as f: File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/rasterio/env.py", line 445, in wrapper return f(args, kwds) File "/home/mtrent/miniconda3/envs/landcover/lib/python3.7/site-packages/rasterio/init.py", line 219, in open s = DatasetReader(path, driver=driver, sharing=sharing, **kwargs) File "rasterio/_base.pyx", line 218, in rasterio._base.DatasetBase.init rasterio.errors.RasterioIOError: '/vsicurl/https://modeloutput.blob.core.windows.net/full-usa-output/1_3_2019/data/v1/2012/states/tx/tx_1m_2012/31097/m_3109762_nw_14_1_20120728_probs.tif' does not exist in the file system, and is not recognized as a supported dataset name.

I'm running this using a WSL with the environment configured according to the installation instructions. I get these errors when running both the NAIP dataset as well as the CVPR Landcover dataset.

[calebrob6]

Hey Michael -- it looks like anonymous access to the pre-computed land cover map was turned off due to a security policy in the container that it is hosted on. Let me see if I can re-enable.

[michaeltrent]

Thanks Caleb! Curious if there is any update on this?

[calebrob6]

Can you try now?

[michaeltrent]

Works now! Thank you!

[michaeltrent]

Hi Caleb,

I'm running into this error once again. Is it possible the access has been turned off once again? Sorry to be a pain.

[michaeltrent]

Is there any update on this?

[calebrob6]

Hey Michael, yes I'm working on it