supermercado (fio collect) is giving warning and generating empty file

manishsharma28 commented 4 years ago

cat aoi.geojson | supermercado burn 19 | mercantile shapes | fio collect > aoi_z19.geojson

Command running with error but an empty file (aoi_z19.geojson) is generated ............................................................................................................................................................................................. The warning message is as following:

Traceback (most recent call last): File "c:\programdata\anaconda3\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "c:\programdata\anaconda3\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\ProgramData\Anaconda3\Scripts\fio.exe__main.py", line 9, in File "c:\programdata\anaconda3\lib\site-packages\click\core.py", line 764, in call__ return self.main(args, kwargs) File "c:\programdata\anaconda3\lib\site-packages\click\core.py", line 717, in main rv = self.invoke(ctx) File "c:\programdata\anaconda3\lib\site-packages\click\core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\programdata\anaconda3\lib\site-packages\click\core.py", line 956, in invoke return ctx.invoke(self.callback, ctx.params) File "c:\programdata\anaconda3\lib\site-packages\click\core.py", line 555, in invoke return callback(args, kwargs) File "c:\programdata\anaconda3\lib\site-packages\click\decorators.py", line 17, in new_func return f(get_current_context(), *args, *kwargs) File "c:\programdata\anaconda3\lib\site-packages\fiona\fio__init__.py", line 18, in wrapper return f(args, kwds) File "c:\programdata\anaconda3\lib\site-packages\fiona\fio\collect.py", line 58, in collect first_line = next(stdin) StopIteration

JmeCS commented 3 years ago

This is happening for me too - any movement on this?

dnomadb commented 3 years ago

Hello @manishsharma28 !

What does the output of

cat aoi.geojson | supermercado burn 19

look like? Also could you provide be with a sample of aoi.geojson?

JmeCS commented 3 years ago

After some additional troubleshooting, it looks like it was unhappy due to a MULTIPOLYGON geometry type in my geojson. @manishsharma28 - if your geojson contains multipolygons, try exploding them into polygons and see if that fixes the problem.

INF800 commented 3 years ago

Hi, any activity on this? I am facing the same issue.

INF800 commented 3 years ago

Hello @manishsharma28 !

What does the output of
cat aoi.geojson | supermercado burn 19
look like? Also could you provide be with a sample of aoi.geojson?

Hi, I have the geojson file which can reproduce the issue. Shall I share?

INF800 commented 3 years ago

@dnomadb

File: trn_aoi.geojson

{
"type": "FeatureCollection",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:EPSG::32643" } },
"features": [
{ "type": "Feature", "properties": { }, "geometry": { "type": "Polygon", "coordinates": [ [ [ 285811.916865957551636, 2123824.416182775981724 ], [ 285847.576544279581867, 2123461.876119835767895 ], [ 285912.952621203206945, 2123033.95997997187078 ], [ 285978.328698126890231, 2122659.533357590902597 ], [ 286097.194292533502448, 2122362.369371574372053 ], [ 286115.024131694517564, 2122261.333616328425705 ], [ 286536.99699183809571, 2122564.440882065333426 ], [ 286424.074677151802462, 2123224.14493102254346 ], [ 286358.698600228119176, 2123491.592518437188119 ], [ 286328.982201626466122, 2123693.664028928615153 ], [ 286222.003166660491843, 2123717.437147810123861 ], [ 286275.492684143478982, 2123925.451938021462411 ], [ 286103.137572253821418, 2123984.884735224768519 ], [ 285936.72574008454103, 2124109.693609351757914 ], [ 285811.916865957551636, 2123824.416182775981724 ] ] ] } }
]
}

command causing warning and generating empty file:

$ cat trn_aoi.geojson | supermercado burn 19 | mercantile shapes | fio collect > trn_aoi_z19tiles.geojson

error/warning:

ERROR 1: PROJ: proj_create_from_database: Open of /opt/conda/envs/solaris/share/proj failed
/opt/conda/envs/solaris/lib/python3.7/site-packages/mercantile/__init__.py:79: FutureWarning: Mercantile 2.0 will require tile x and y to be within the range (0, 2 ** zoom)
  FutureWarning,
Traceback (most recent call last):
  File "/opt/conda/envs/solaris/bin/fio", line 11, in <module>
    sys.exit(main_group())
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/fiona/fio/__init__.py", line 18, in wrapper
    return f(*args, **kwds)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/fiona/fio/collect.py", line 58, in collect
    first_line = next(stdin)
StopIteration

INF800 commented 3 years ago

After some additional troubleshooting, it looks like it was unhappy due to a MULTIPOLYGON geometry type in my geojson. @manishsharma28 - if your geojson contains multipolygons, try exploding them into polygons and see if that fixes the problem.

I tried exploding. But it does not seem to work.

sgillies commented 3 years ago

@INF800 thanks for posting your data file. In it I see the following

"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:EPSG::32643" } }

Your data is not strict GeoJSON. Supermercado requires your coordinates to use the standard OGC:CRS84 system (longitude and latitude using WGS84). You must reproject your data from EPSG:32643 to OGC:CRS84.

INF800 commented 3 years ago

@sgillies It is NOT working (or am I making some mistake?). Thankyou for responding.

Detailed description:

I used these options to export the polygon using QGIS

And the file after exporting looks like this: aoi_ogc_crs_84.geojson

{
"type": "FeatureCollection",
"name": "aoi_ogc_crs_84",
"features": [
{ "type": "Feature", "properties": { "id": 0 }, "geometry": { "type": "MultiPolygon", "coordinates": [ [ [ [ 286093.184368180751335, 2122448.917276436462998 ], [ 285839.117529983399436, 2123632.333864882122725 ], [ 286340.56523695186479, 2123665.763712013605982 ], [ 286474.284625476808287, 2122512.433985985815525 ], [ 286093.184368180751335, 2122448.917276436462998 ] ] ] ] } },
{ "type": "Feature", "properties": { "id": 1 }, "geometry": { "type": "MultiPolygon", "coordinates": [ [ [ [ 286514.400442034238949, 2122529.148909551557153 ], [ 286360.623145230580121, 2123662.420727300457656 ], [ 286731.694448387250304, 2123672.449681439902633 ], [ 286915.558607609069441, 2122689.612175781745464 ], [ 286514.400442034238949, 2122529.148909551557153 ] ] ] ] } }
]
}

Now, I want to burn the polygon with id==0 - which is saved in trn_aoi.geojson. So I use:

aoi_df = gpd.read_file('/src/data/thane/aoi_ogc_crs_84.geojson').explode()
aoi_df[aoi_df['id']==0].to_crs("OGC:CRS84").to_file('trn_aoi.geojson', driver='GeoJSON')

trn_aoi.geojson:

{
"type": "FeatureCollection",
"features": [
{ "type": "Feature", "properties": { "level_0": 0, "level_1": 0, "id": 0 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ 286093.184368180751335, 2122448.917276436462998 ], [ 285839.117529983399436, 2123632.333864882122725 ], [ 286340.56523695186479, 2123665.763712013605982 ], [ 286474.284625476808287, 2122512.433985985815525 ], [ 286093.184368180751335, 2122448.917276436462998 ] ] ] } }
]
}

Finally I run the burn command which results in error

$ cat trn_aoi.geojson | supermercado burn 19 | mercantile shapes | fio collect

ERROR 1: PROJ: proj_create_from_database: Open of /opt/conda/envs/solaris/share/proj failed
/opt/conda/envs/solaris/lib/python3.7/site-packages/mercantile/__init__.py:79: FutureWarning: Mercantile 2.0 will require tile x and y to be within the range (0, 2 ** zoom)
  FutureWarning,
Traceback (most recent call last):
  File "/opt/conda/envs/solaris/bin/fio", line 11, in <module>
    sys.exit(main_group())
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/fiona/fio/__init__.py", line 18, in wrapper
    return f(*args, **kwds)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/fiona/fio/collect.py", line 58, in collect
    first_line = next(stdin)
StopIteration

INF800 commented 3 years ago

Please note that ERROR 1: PROJ: proj_create_from_database: Open of /opt/conda/envs/solaris/share/proj failed has nothing to do with the issue because in another case (with another geojson file), the same appoach gave expected results even with the above ERROR 1

INF800 commented 3 years ago

@dnomadb

File: trn_aoi.geojson

{
"type": "FeatureCollection",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:EPSG::32643" } },
"features": [
{ "type": "Feature", "properties": { }, "geometry": { "type": "Polygon", "coordinates": [ [ [ 285811.916865957551636, 2123824.416182775981724 ], [ 285847.576544279581867, 2123461.876119835767895 ], [ 285912.952621203206945, 2123033.95997997187078 ], [ 285978.328698126890231, 2122659.533357590902597 ], [ 286097.194292533502448, 2122362.369371574372053 ], [ 286115.024131694517564, 2122261.333616328425705 ], [ 286536.99699183809571, 2122564.440882065333426 ], [ 286424.074677151802462, 2123224.14493102254346 ], [ 286358.698600228119176, 2123491.592518437188119 ], [ 286328.982201626466122, 2123693.664028928615153 ], [ 286222.003166660491843, 2123717.437147810123861 ], [ 286275.492684143478982, 2123925.451938021462411 ], [ 286103.137572253821418, 2123984.884735224768519 ], [ 285936.72574008454103, 2124109.693609351757914 ], [ 285811.916865957551636, 2123824.416182775981724 ] ] ] } }
]
}

command causing warning and generating empty file:

$ cat trn_aoi.geojson | supermercado burn 19 | mercantile shapes | fio collect > trn_aoi_z19tiles.geojson

error/warning:

ERROR 1: PROJ: proj_create_from_database: Open of /opt/conda/envs/solaris/share/proj failed
/opt/conda/envs/solaris/lib/python3.7/site-packages/mercantile/__init__.py:79: FutureWarning: Mercantile 2.0 will require tile x and y to be within the range (0, 2 ** zoom)
  FutureWarning,
Traceback (most recent call last):
  File "/opt/conda/envs/solaris/bin/fio", line 11, in <module>
    sys.exit(main_group())
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/fiona/fio/__init__.py", line 18, in wrapper
    return f(*args, **kwds)
  File "/opt/conda/envs/solaris/lib/python3.7/site-packages/fiona/fio/collect.py", line 58, in collect
    first_line = next(stdin)
StopIteration

@sgillies please ignore the message. everything is working as expected.

I did not know espg:4326/wgs84 and urn:ogc:def:crs:ogc:1.3:crs84 are both same

sgillies commented 3 years ago

ERROR 1: PROJ: proj_create_from_database: Open of /opt/conda/envs/solaris/share/proj failed indicates that your conda environment is not functional, probably due to conflicting packages. That's not a problem with supermercado.

mapbox / supermercado

supermercado (fio collect) is giving warning and generating empty file #38

Detailed description: