False negative for geometry type

[NathanEne]

Some datasets such as water-rights-applications-public are composed of two original datasets one without geometry and one with. They are then stacked on top of each other. This can lead to all the geometry being at the bottom of the dataset so the method currently being used to check for geometry type with the first request can fail. One example solution that has been working well for me is within PR 157 and includes a bottom check aswell as a check at the top. This issue could also possibly be solved with the --query command but that was not initially obvious to me until I realized what was happening so figured it might be nice for some additional functionality

[smnorris]

Thanks for the contribution!

Example error for reference:

$ bc2pg water-rights-applications-public
2024-01-28 09:35:24,493:INFO:bcdata.wfs: Total features requested: 27826
2024-01-28 09:36:06,285:INFO:bcdata.wfs: https://openmaps.gov.bc.ca/geo/pub/wfs?service=WFS&version=2.0.0&request=GetFeature&typeName=WHSE_WATER_MANAGEMENT.WLS_WATER_RIGHTS_APPLICTNS_SV&outputFormat=json&SRSNAME=epsg%3A3005&sortby=OBJECTID&startIndex=0&count=10000
Traceback (most recent call last):
  File "/Users/snorris/Projects/repo/bcdata/bcdata_env/bin/bcdata", line 8, in <module>
    sys.exit(cli())
  File "/Users/snorris/Projects/repo/bcdata/bcdata_env/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/snorris/Projects/repo/bcdata/bcdata_env/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/snorris/Projects/repo/bcdata/bcdata_env/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/snorris/Projects/repo/bcdata/bcdata_env/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/snorris/Projects/repo/bcdata/bcdata_env/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/snorris/Projects/repo/bcdata/bcdata/cli.py", line 381, in bc2pg
    out_table = bcdata.bc2pg(
  File "/Users/snorris/Projects/repo/bcdata/bcdata/bc2pg.py", line 105, in bc2pg
    geometry_type = geometry_type.upper()
AttributeError: 'NoneType' object has no attribute 'upper'

[smnorris]

Fixed with https://github.com/smnorris/bcdata/pull/157 , https://github.com/smnorris/bcdata/pull/159

[smnorris]

I thought this seemed familiar - we added the bc2pg --geometry_type option as a workaround earlier https://github.com/smnorris/bcdata/issues/132 for known layers / geometry types.

But the additional sniff seems to work well as a default.

[NathanEne]

ah using that --geometry_type flag must have gotten removed from our pipelines recently in a refactor