search

Matico-Platform / pmtiles-utils

A command line tool to convert any GDAL support geospatial data file to the PMTiles, a low-cost cloud-optimized tile format.
16 stars 2 forks source link

pyogrio v0.4.0 is currently using GDAL 3.4.1 #2

Open nmarchio opened 2 years ago

nmarchio commented 2 years ago

Hey @nofurtherinformation ! Looks like this is a documented issue in pyogrio's repo as of early last month: pyogrio v0.4.0 is currently using GDAL 3.4.1 instead of 3.5+ which supports GeoParquet. This would explain the issues I've been having trying to use pyogrio with GeoParquet file formats.

Here is the relevant comment from Joris about the issue: https://github.com/geopandas/pyogrio/issues/154#issuecomment-1236129245

And here is the error that tipped me off:

(tiling) nm@nmarchio pmtiles-utils % python3 convert --input=subsaharan_regions.parquet --output=output.pmtiles --config=tiles_config.yaml --retain=TRUE 
Force Overwrite [True]: True
Recycling old files...
šŸ”„ Starting File Conversion šŸ”„
Checking ogr2ogr / gdal compatibility...
Gdal supports FlatGeoBuf!
Checking Tippecanoe Compatibility...
Tippecanoe supports FlatGeoBuf!
Your Tippecanoe and Ogr2ogr install are compatible!
Converting input file to FGB format...
šŸ”„ Converting to flatgeobuf šŸ”„
ERROR 4: `./subsaharan_regions.parquet' not recognized as a supported file format.
Traceback (most recent call last):
  File "pyogrio/_io.pyx", line 132, in pyogrio._io.ogr_open
  File "pyogrio/_err.pyx", line 177, in pyogrio._err.exc_wrap_pointer
pyogrio._err.CPLE_OpenFailedError: './subsaharan_regions.parquet' not recognized as a supported file format.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/nm/miniconda3/envs/tiling/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/nm/miniconda3/envs/tiling/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/nm/Desktop/pmtiles-utils/convert/__main__.py", line 3, in <module>
    cli_convert()
  File "/Users/nm/miniconda3/envs/tiling/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/nm/miniconda3/envs/tiling/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/nm/miniconda3/envs/tiling/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/nm/miniconda3/envs/tiling/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/nm/Desktop/pmtiles-utils/convert/cli.py", line 45, in cli_convert
    fgb_success = convert_to_fgb(input, output_name) 
  File "/Users/nm/Desktop/pmtiles-utils/convert/geo_file_utils.py", line 15, in convert_to_fgb
    df = pyogrio.read_dataframe(f"./{input}")
  File "/Users/nm/miniconda3/envs/tiling/lib/python3.10/site-packages/pyogrio/geopandas.py", line 137, in read_dataframe
    meta, index, geometry, field_data = read(
  File "/Users/nm/miniconda3/envs/tiling/lib/python3.10/site-packages/pyogrio/raw.py", line 120, in read
    result = ogr_read(
  File "pyogrio/_io.pyx", line 866, in pyogrio._io.ogr_read
  File "pyogrio/_io.pyx", line 141, in pyogrio._io.ogr_open
pyogrio.errors.DataSourceError: './subsaharan_regions.parquet' not recognized as a supported file format.
nmarchio commented 2 years ago

Out of curiosity, I wonder how you were able to get this to work? Did you just convert the parquet to gpkg with QGIS then use this repo? Were you able to run at all with GeoParquet in any of your testing?

Also looks like this is part of a bigger issue related to homebrew GDAL installation not including parquet.

https://github.com/opengeospatial/geoparquet/discussions/99

You can see here I'm using 3.5.2 and no parquet support:

image
nmarchio commented 2 years ago

And for vis, filed an issue here to get GDAL 3.5+ added to QGIS: https://github.com/qgis/QGIS-Mac-Packager/issues/156