stevenpawley / Pyspatialml

Machine learning modelling for spatial data
GNU General Public License v3.0
145 stars 29 forks source link

Raster stack vrts? should they work, or expected behaviour? #33

Closed RichardScottOZ closed 3 years ago

RichardScottOZ commented 3 years ago
predictors_list.append(r'J:\McArthur\sentinel endmembers\onelib.vrt')  #need to convert to tiff rio commandline job
stack = Raster(predictors_list)
---------------------------------------------------------------------------
CPLE_OpenFailedError                      Traceback (most recent call last)
rasterio\_base.pyx in rasterio._base.DatasetBase.__init__()

rasterio\_shim.pyx in rasterio._shim.open_dataset()

rasterio\_err.pyx in rasterio._err.exc_wrap_pointer()

CPLE_OpenFailedError: 'J:/McArthur/sentinel endmembers/onelib.vrt' not recognized as a supported file format.

During handling of the above exception, another exception occurred:

RasterioIOError                           Traceback (most recent call last)
<ipython-input-3-ab6a1e11ff77> in <module>
      8 
      9 predictors_list.append(r'J:\McArthur\sentinel endmembers\onelib.vrt')
---> 10 stack = Raster(predictors_list)

~\AppData\Local\Continuum\anaconda3\envs\stackstac\lib\site-packages\pyspatialml\raster.py in __init__(self, src, crs, transform, nodata, mode, file_path, driver, tempdir, in_memory)
    428         elif isinstance(src, list) and all(isinstance(x, str) for x in src):
    429             for f in src:
--> 430                 r = rasterio.open(f, mode=mode, driver=driver)
    431                 for i in range(r.count):
    432                     band = rasterio.band(r, i + 1)

~\AppData\Local\Continuum\anaconda3\envs\stackstac\lib\site-packages\rasterio\env.py in wrapper(*args, **kwds)
    433 
    434         with env_ctor(session=session):
--> 435             return f(*args, **kwds)
    436 
    437     return wrapper

~\AppData\Local\Continuum\anaconda3\envs\stackstac\lib\site-packages\rasterio\__init__.py in open(fp, mode, driver, width, height, count, crs, transform, dtype, nodata, sharing, **kwargs)
    205         # None.
    206         if mode == 'r':
--> 207             s = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
    208         elif mode == "r+":
    209             s = get_writer_for_path(path, driver=driver)(

rasterio\_base.pyx in rasterio._base.DatasetBase.__init__()

RasterioIOError: 'J:/McArthur/sentinel endmembers/onelib.vrt' not recognized as a supported file format.
f = r'J:\McArthur\sentinel endmembers\onelib.vrt'
sentinel_endmembers = rioxarray.open_rasterio(f, chunks=(1, 8000,8000))
<xarray.DataArray (band: 16, y: 81795, x: 78064)>
dask.array<open_rasterio-e3df6e6acedc4f31ce35af45c4e67ea5<this-array>, shape=(16, 81795, 78064), dtype=int16, chunksize=(1, 8000, 8000)>
Coordinates:
  * band         (band) int32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
  * y            (y) float64 -11.89 -11.89 -11.89 ... -19.24 -19.24 -19.24
  * x            (x) float64 132.1 132.1 132.1 132.1 ... 139.2 139.2 139.2 139.2
    spatial_ref  int32 0
Attributes:
    STATISTICS_APPROXIMATE:    YES
    STATISTICS_MAXIMUM:        172
    STATISTICS_MEAN:           6.2712016463438
    STATISTICS_MINIMUM:        0
    STATISTICS_STDDEV:         16.48841430401
    STATISTICS_VALID_PERCENT:  100
    scale_factor:              1.0
    add_offset:                0.0
    grid_mapping:              spatial_ref
stevenpawley commented 3 years ago

Sorry, this has taken me a while to look at. Should be fixed with https://github.com/stevenpawley/Pyspatialml/commit/6234f83b1291f091d1fc3db89841f2de8e6b9d45

RichardScottOZ commented 3 years ago

No worries, not urgent - if can use vrts, taking double advantage of on-diskiness I guess!

I did some really large scale inference with this recently.