search

ckan / ckanext-spatial

Geospatial extension for CKAN
http://docs.ckan.org/projects/ckanext-spatial
125 stars 192 forks source link

Installation instructions for Docker #253

Open guilhermeiablo opened 3 years ago

guilhermeiablo commented 3 years ago

I'm following the instructions provided at the CKAN documentation for 2.9.3 for installing extensions with docker-compose, since it uses ckanext-spatial as an example. I'm not able to complete the process though.

The problems begin right after the documentation states that "Some extensions require database upgrades, often through paster scripts. E.g., ckanext-spatial".

Then comes two lines of code that do not work as they should:

docker exec -it db psql -U ckan -f 20_postgis_permissions.sql This first one throws a "no such file 20_postgis_permissions.sql" error.

I managed to fix this by asking for direction at the gitter channel. So it got changed to this:

docker exec -it db psql -U ckan -f /docker-entrypoint-initdb.d/20_postgis_permissions.sql The second one though, still not working:

docker exec -it ckan /usr/local/bin/ckan-paster --plugin=ckanext-spatial spatial initdb -c /etc/ckan/production.ini It gives the error:

OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "/usr/local/bin/ckan-paster": stat /usr/local/bin/ckan-paster: no such file or directory: unknown

EvgeniaPatsoni commented 3 years ago

Try the following please, it worked for me:

After executing docker exec -it db psql -U ckan -f /docker-entrypoint-initdb.d/20_postgis_permissions.sql

  1. docker exec -it ckan /bin/bash
  2. source $CKAN_VENV/bin/activate && cd $CKAN_VENV/src/
  3. paster --plugin=ckanext-spatial spatial initdb --config=/etc/ckan/production.ini
guilhermeiablo commented 3 years ago

I tried that but still no luck. From gitter, I learned that paster commands are no longer valid since Ckan 2.9

What worked for me in the end was this (I'm running docker on a mac, so I have ran into some specificities):

  1. Update permissions for postgis from host docker exec -it db psql -U ckan -f /docker-entrypoint-initdb.d/20_postgis_permissions.sql

  2. Access Docker Linux VM: $ docker run -it --privileged --pid=host justincormack/nsenter1

  3. From inside the VM: 

    $ vi var/lib/docker/volumes/docker_ckan_config/_data/production.ini 
add spatial_metadata spatial_query to ckan.plugins
add ckanext.spatial.search_backend = solr

  4. Restart container: $ docker-compose restart ckan

​​5. Enter the running ckan container: docker exec -it ckan /bin/bash -c "export TERM=xterm; exec bash"

  1. Inside the running container, activate the virtualenv source $CKAN_VENV/bin/activate && cd $CKAN_VENV/src/
  2. Spatial initdb ckan --config=/etc/ckan/production.ini spatial initdb
  3. Restart container: $ docker-compose restart ckan
ccancellieri commented 2 years ago

could be of some interest https://github.com/okfn/docker-ckan/issues/84

rmeissn commented 2 years ago

I'm setting up a compose ckan deployment (see https://gitlab.com/infai/center-of-competence/efficient-technology-integration/ada-alkis-datenbeschaffungs-api/ada-infrastructure/-/tree/ckanext-spatial ) and I am struggling with the installation instructions for ckanext-spatial, which are somewhat outdated.

I've changed from postgres/postgres:12 to postgis/postgis:12 for the database, installed the extension via https://gitlab.com/infai/center-of-competence/efficient-technology-integration/ada-alkis-datenbeschaffungs-api/ada-infrastructure/-/blob/ckanext-spatial/CKAN/Dockerfile#L76 , added the plugins to the ini file and added ckan spatial initdb to the ckan entrypoint script (see last commit).

Unfortunately, ckan won't start up and raises:

2022-08-16 12:47:51,274 INFO  [ckan.cli] Using configuration file /etc/ckan/ckan.ini
2022-08-16 12:47:51,274 INFO  [ckan.config.environment] Loading static files from public
2022-08-16 12:47:51,291 INFO  [ckan.config.environment] Loading templates from /venv/src/ckan/ckan/templates
2022-08-16 12:47:51,521 INFO  [ckan.config.environment] Loading templates from /venv/src/ckan/ckan/templates
2022-08-16 12:47:51,553 DEBUG [ckanext.harvest.model] Harvest tables defined in memory
2022-08-16 12:47:51,558 DEBUG [ckanext.harvest.model] Harvest tables already exist
2022-08-16 12:47:51,564 DEBUG [ckanext.spatial.plugin] Setting up the spatial model
Traceback (most recent call last):
  File "/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1243, in _execute_context
    self.dialect.do_execute(
  File "/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 550, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedFunction: function postgis_lib_version() does not exist
LINE 1: SELECT postgis_lib_version()
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/ckan", line 33, in <module>
    sys.exit(load_entry_point('ckan', 'console_scripts', 'ckan')())
  File "/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 781, in main
    with self.make_context(prog_name, args, **extra) as ctx:
  File "/venv/lib/python3.8/site-packages/click/core.py", line 700, in make_context
    self.parse_args(ctx, args)
  File "/venv/src/ckan/ckan/cli/cli.py", line 115, in parse_args
    result = super(ExtendableGroup, self).parse_args(ctx, args)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1212, in parse_args
    rest = Command.parse_args(self, ctx, args)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1048, in parse_args
    value, args = param.handle_parse_result(ctx, opts, args)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1630, in handle_parse_result
    value = invoke_param_callback(self.callback, ctx, self, value)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 123, in invoke_param_callback
    return callback(ctx, param, value)
  File "/venv/src/ckan/ckan/cli/cli.py", line 125, in _init_ckan_config
    _add_ctx_object(ctx, value)
  File "/venv/src/ckan/ckan/cli/cli.py", line 134, in _add_ctx_object
    ctx.obj = CtxObject(path)
  File "/venv/src/ckan/ckan/cli/cli.py", line 56, in __init__
    self.app = make_app(self.config)
  File "/venv/src/ckan/ckan/config/middleware/__init__.py", line 56, in make_app
    load_environment(conf)
  File "/venv/src/ckan/ckan/config/environment.py", line 123, in load_environment
    p.load_all()
  File "/venv/src/ckan/ckan/plugins/core.py", line 165, in load_all
    load(*plugins)
  File "/venv/src/ckan/ckan/plugins/core.py", line 193, in load
    plugins_update()
  File "/venv/src/ckan/ckan/plugins/core.py", line 153, in plugins_update
    environment.update_config()
  File "/venv/src/ckan/ckan/config/environment.py", line 309, in update_config
    plugin.configure(config)
  File "/venv/src/ckanext-spatial/ckanext/spatial/plugin/__init__.py", line 88, in configure
    setup_model()
  File "/venv/src/ckanext-spatial/ckanext/spatial/model/package_extent.py", line 22, in setup
    define_spatial_tables(srid)
  File "/venv/src/ckanext-spatial/ckanext/spatial/model/package_extent.py", line 69, in define_spatial_tables
    package_extent_table = setup_spatial_table(PackageExtent, db_srid)
  File "/venv/src/ckanext-spatial/ckanext/spatial/geoalchemy_common.py", line 68, in setup_spatial_table
    management = (postgis_version()[:1] == '1')
  File "/venv/src/ckanext-spatial/ckanext/spatial/geoalchemy_common.py", line 40, in postgis_version
    result = Session.execute('SELECT postgis_lib_version()')
  File "/venv/lib/python3.8/site-packages/sqlalchemy/orm/scoping.py", line 162, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1268, in execute
    return self._connection_for_bind(bind, close_with_result=True).execute(
  File "/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1101, in _execute_clauseelement
    ret = self._execute_context(
  File "/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1247, in _execute_context
    self._handle_dbapi_exception(
  File "/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 399, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
    raise value.with_traceback(tb)
  File "/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1243, in _execute_context
    self.dialect.do_execute(
  File "/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 550, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) function postgis_lib_version() does not exist
LINE 1: SELECT postgis_lib_version()
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

[SQL: SELECT postgis_lib_version()]
(Background on this error at: http://sqlalche.me/e/f405)

Might this be related to not having fuzzystrmatch installed with the postgis/postgis image (e.g. https://hub.docker.com/r/mdillon/postgis/# installed it, but the image is rather outdated)? I guess that postgis itself is correctly set up, as I use a specialized postgis image. Unfortunately, the postgis install instructions (e.g. the sql files) do not match the current postgis images any more...