smnorris / bcdata

Python and command line tools for quick access to DataBC geo-data available via WFS/WCS.
MIT License
29 stars 7 forks source link

bc2pg - assessment watersheds fail to load #102

Closed smnorris closed 2 years ago

smnorris commented 2 years ago

All other tables seem fine.

Version bcdata 0.6.1

OS NAME="Ubuntu" VERSION="20.04.4 LTS (Focal Fossa)"

Postgres 13.4

                                                                         postgis_full_version
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
 POSTGIS="3.1.3 008d2db" [EXTENSION] PGSQL="130" GEOS="3.8.0-CAPI-1.13.1 " PROJ="6.3.1" LIBXML="2.9.10" LIBJSON="0.13.1" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)"

Error

$ bcdata bc2pg whse_basemapping.fwa_assessment_watersheds_poly --schema fwapg --table fwa_assessment_watersheds_poly_load
INFO:bcdata.cli:ogr2ogr -f PostgreSQL PG:host=localhost user=snorris dbname=hillcrestgeo port=5432 -t_srs EPSG:3005 -nln fwapg.fwa_assessment_watersheds_poly_load https://openmaps.gov.bc.ca/geo/pub/wfs?service=WFS&version=2.0.0&request=GetFeature&typeName=WHSE_BASEMAPPING.FWA_ASSESSMENT_WATERSHEDS_POLY&outputFormat=json&SRSNAME=epsg%3A4326&sortby=OBJECTID&startIndex=0&count=10000 -overwrite -lco GEOMETRY_NAME=geom -lco UNLOGGED=ON -lco SPATIAL_INDEX=NONE
Traceback (most recent call last):
  File "/home/snorris/.local/lib/python3.8/site-packages/bcdata/cli.py", line 429, in bc2pg
    db.execute(dbq)
  File "/home/snorris/.local/lib/python3.8/site-packages/bcdata/database.py", line 75, in execute
    result = curs.execute(sql, params)
psycopg2.errors.UndefinedTable: relation "fwapg.fwa_assessment_watersheds_poly_load" does not exist
LINE 3:                     (LIKE "fwapg"."fwa_assessment_watersheds...
                                  ^

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/snorris/.local/bin/bcdata", line 8, in <module>
    sys.exit(cli())
  File "/home/snorris/.local/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/snorris/.local/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/snorris/.local/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/snorris/.local/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/snorris/.local/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/snorris/.local/lib/python3.8/site-packages/bcdata/cli.py", line 487, in bc2pg
    raise RuntimeError("Loading to or from temp tables failed")
RuntimeError: Loading to or from temp tables failed
make: *** [Makefile:103: .make/fwa_assessment_watersheds_poly] Error 1
smnorris commented 2 years ago

Named watersheds also fails. Very odd.

smnorris commented 2 years ago

memory issue, use smaller chunks