Make failure with latest osm2pgsql-replication

rustprooflabs commented 1 year ago

What version of PgOSM Flex are you using?

Building from latest main branch following steps here for ensuring full updates.

What did you do exactly?

docker pull postgis/postgis:15-3.3
docker build --no-cache -t rustprooflabs/pgosm-flex .
time make

What did you expect to happen?

The make command would complete successfully.

What did happen instead?

Truncated output, including versions used.

2023-08-15 17:50:34,308:INFO:pgosm-flex:pgosm_flex:Running osm2pgsql
2023-08-15 17:50:34,318:INFO:pgosm-flex:helpers:2023-08-15 17:50:34  osm2pgsql version 1.9.0 (1.9.0)
2023-08-15 17:50:34,323:INFO:pgosm-flex:helpers:2023-08-15 17:50:34  Database version: 15.4 (Debian 15.4-1.pgdg110+1)
2023-08-15 17:50:34,323:INFO:pgosm-flex:helpers:2023-08-15 17:50:34  PostGIS version: 3.3
2023-08-15 17:50:34,324:INFO:pgosm-flex:helpers:2023-08-15 17:50:34  Storing properties to table '"public"."osm2pgsql_properties"'.

...

# NOTE: This step tests --replication file for an initial load.

...

2023-08-15 17:47:44,519:INFO:pgosm-flex:pgosm_flex:Skipping calculating nested polygons
2023-08-15 17:47:44,646:INFO:pgosm-flex:helpers:Traceback (most recent call last):
2023-08-15 17:47:44,646:INFO:pgosm-flex:helpers:File "/usr/local/bin/osm2pgsql-replication", line 724, in <module>
2023-08-15 17:47:44,646:INFO:pgosm-flex:helpers:sys.exit(main())
2023-08-15 17:47:44,646:INFO:pgosm-flex:helpers:File "/usr/local/bin/osm2pgsql-replication", line 704, in main
2023-08-15 17:47:44,646:INFO:pgosm-flex:helpers:with DBConnection(args) as db:
2023-08-15 17:47:44,646:INFO:pgosm-flex:helpers:File "/usr/local/bin/osm2pgsql-replication", line 125, in __init__
2023-08-15 17:47:44,647:INFO:pgosm-flex:helpers:self.name = self.conn.get_dsn_parameters()['dbname']
2023-08-15 17:47:44,647:INFO:pgosm-flex:helpers:AttributeError: 'Connection' object has no attribute 'get_dsn_parameters'
2023-08-15 17:47:44,667:ERROR:pgosm-flex:pgosm_flex:Failed to run osm2pgsql-replication. Return code: 1
Failed to run osm2pgsql-replication. Return code: 1 - Check the log output for details.
make: *** [Makefile:138: docker-exec-replication-w-input-file] Error 1

real    0m44.274s
user    0m0.316s
sys 0m0.198s

What did you do to try analyzing the problem?

Verified the issue is scoped to the --replication functionality and isn't a wider problem. Running the simpler unit tests suggested in the Makefile succeeds.

time make docker-exec-default unit-tests

truncated output

----------------------------------------------------------------------
Ran 49 tests in 2.050s

OK

...
Running PgOSM-Flex test queries
Data output tests completed successfully.

real    0m58.064s
user    0m0.252s
sys 0m0.227s

rustprooflabs commented 1 year ago

The docker exec command involved is:

docker exec -it \
    -e POSTGRES_PASSWORD=mysecretpassword \
    -e POSTGRES_USER=postgres \
    -u 1000:1000 \
    pgosm python3 docker/pgosm_flex.py  \
    --layerset=minimal \
    --ram=2 \
    --replication \
    --input-file=/app/output/custom_source_file.osm.pbf \
    --skip-qgis-style --skip-nested

Python versions that might matter here:

docker exec -it pgosm /bin/bash -c "pip freeze | grep psycopg"

psycopg==3.1.10
psycopg-binary==3.1.10

rustprooflabs commented 1 year ago

Fixed via https://github.com/openstreetmap/osm2pgsql/pull/2044

rustprooflabs / pgosm-flex