geometalab / osmaxx

OpenStreetMap Arbitrary Excerpt Export - Get the OpenStreetMap data you want in the file format you need
http://osmaxx.hsr.ch/
MIT License
26 stars 3 forks source link

GIS exports broken due to translit from giggls/mapnik-german-l10n master #870

Open das-g opened 7 years ago

das-g commented 7 years ago

The master branch of giggls/mapnik-german-l10n (currently giggls/mapnik-german-l10n@40e107f987461eea8929a8cb1f940f7c3d35e894 a.k.a. v2.2.5) has changed in ways that break our current use of it: Exporting GIS formats with a freshly built worker docker image fails with

exception caught while processing /home/py/osmaxx/conversion/converters/converter_gis/bootstrap/sql/functions/0030_transliterate.sql
11:18:28 sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) required extension "unaccent" is not installed
 [SQL: "---------------------------------------------------------------------------------------\n-- write or print (a letter or word) using the closest corresponding letters of\n-- a different alphabet or language.\n-- EG. select osml10n_translit('Москва́');\n--     Moskvá\n---------------------------------------------------------------------------------------\nCREATE EXTENSION IF NOT EXISTS postgis;\nCREATE EXTENSION IF NOT EXISTS osml10n;\n"]
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
psycopg2.ProgrammingError: required extension "unaccent" is not installed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/rq/worker.py", line 700, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.4/site-packages/rq/job.py", line 500, in perform
    self._result = self.func(*self.args, **self.kwargs)
  File "/home/py/osmaxx/conversion/converters/converter.py", line 43, in convert
    converter.perform_export(**params)
  File "/home/py/osmaxx/conversion/converters/converter_gis/gis.py", line 32, in perform_export
    gis.create_gis_export()
  File "/home/py/osmaxx/conversion/converters/converter_gis/gis.py", line 75, in create_gis_export
    _bootstrapper.bootstrap()
  File "/home/py/osmaxx/conversion/converters/converter_gis/bootstrap/bootstrap.py", line 29, in bootstrap
    self._setup_db_functions()
  File "/home/py/osmaxx/conversion/converters/converter_gis/bootstrap/bootstrap.py", line 54, in _setup_db_functions
    self._execute_sql_scripts_in_folder(os.path.join(self._script_base_dir, 'sql', 'functions'))
  File "/home/py/osmaxx/conversion/converters/converter_gis/bootstrap/bootstrap.py", line 111, in _execute_sql_scripts_in_folder
    self._postgres.execute_sql_file(script_path)
  File "/home/py/osmaxx/conversion/converters/converter_gis/helper/postgres_wrapper.py", line 27, in execute_sql_file
    return self.execute_sql_command(psql_command_file.read())
  File "/home/py/osmaxx/conversion/converters/converter_gis/helper/postgres_wrapper.py", line 35, in execute_sql_command
    result = connection.execute(sqlalchemy.text(sql))
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) required extension "unaccent" is not installed
 [SQL: "---------------------------------------------------------------------------------------\n-- write or print (a letter or word) using the closest corresponding letters of\n-- a different alphabet or language.\n-- EG. select osml10n_translit('Москва́');\n--     Moskvá\n---------------------------------------------------------------------------------------\nCREATE EXTENSION IF NOT EXISTS postgis;\nCREATE EXTENSION IF NOT EXISTS osml10n;\n"]
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
psycopg2.ProgrammingError: required extension "unaccent" is not installed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/rq/worker.py", line 700, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.4/site-packages/rq/job.py", line 500, in perform
    self._result = self.func(*self.args, **self.kwargs)
  File "/home/py/osmaxx/conversion/converters/converter.py", line 43, in convert
    converter.perform_export(**params)
  File "/home/py/osmaxx/conversion/converters/converter_gis/gis.py", line 32, in perform_export
    gis.create_gis_export()
  File "/home/py/osmaxx/conversion/converters/converter_gis/gis.py", line 75, in create_gis_export
    _bootstrapper.bootstrap()
  File "/home/py/osmaxx/conversion/converters/converter_gis/bootstrap/bootstrap.py", line 29, in bootstrap
    self._setup_db_functions()
  File "/home/py/osmaxx/conversion/converters/converter_gis/bootstrap/bootstrap.py", line 54, in _setup_db_functions
    self._execute_sql_scripts_in_folder(os.path.join(self._script_base_dir, 'sql', 'functions'))
  File "/home/py/osmaxx/conversion/converters/converter_gis/bootstrap/bootstrap.py", line 111, in _execute_sql_scripts_in_folder
    self._postgres.execute_sql_file(script_path)
  File "/home/py/osmaxx/conversion/converters/converter_gis/helper/postgres_wrapper.py", line 27, in execute_sql_file
    return self.execute_sql_command(psql_command_file.read())
  File "/home/py/osmaxx/conversion/converters/converter_gis/helper/postgres_wrapper.py", line 35, in execute_sql_command
    result = connection.execute(sqlalchemy.text(sql))
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) required extension "unaccent" is not installed
 [SQL: "---------------------------------------------------------------------------------------\n-- write or print (a letter or word) using the closest corresponding letters of\n-- a different alphabet or language.\n-- EG. select osml10n_translit('Москва́');\n--     Moskvá\n---------------------------------------------------------------------------------------\nCREATE EXTENSION IF NOT EXISTS postgis;\nCREATE EXTENSION IF NOT EXISTS osml10n;\n"]
11:18:28 Invoking exception handler <bound method Worker.move_to_failed_queue of <rq.worker.Worker object at 0x7fa9c2acaf98>>
11:18:28 Moving job to 'failed' queue

The last published build of the container uses giggls/mapnik-german-l10n@704ad2af30cae74891da5931f4abbcceb7a8a75d a.k.a. v2.2.0

$> docker run -t --rm geometalab/osmaxx-worker:v3.10.0 bash -c 'cd $CODE/mapnik-german-l10n && git describe --tags && git rev-parse HEAD'
[ ok ] Starting PostgreSQL 9.4 database server: main.
v2.2.0
704ad2af30cae74891da5931f4abbcceb7a8a75d