Latest tag/release prevents ckan from starting

[rmeissn]

I've installed ckan v2.9.5 in a docker environment with python3 and wanted to install ckanext-dcat. I've chose to go with the latest available tags (ckanext-dcat:v1.1.3 and ckanext-harvest:v1.4.0) to get something more stable than from the main branch and prepared a correct ckan.ini, as described in this repo. Unfortunately, CKAN fails on startup with:

root@...# ckan run --host 0.0.0.0
2022-05-04 07:37:37,422 INFO  [ckan.cli] Using configuration file /etc/ckan/ckan.ini
2022-05-04 07:37:37,422 INFO  [ckan.config.environment] Loading static files from public
2022-05-04 07:37:37,449 INFO  [ckan.config.environment] Loading templates from /venv/src/ckan/ckan/templates
Traceback (most recent call last):
  File "/usr/local/bin/ckan", line 33, in <module>
    sys.exit(load_entry_point('ckan', 'console_scripts', 'ckan')())
  File "/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 781, in main
    with self.make_context(prog_name, args, **extra) as ctx:
  File "/venv/lib/python3.8/site-packages/click/core.py", line 700, in make_context
    self.parse_args(ctx, args)
  File "/venv/src/ckan/ckan/cli/cli.py", line 115, in parse_args
    result = super(ExtendableGroup, self).parse_args(ctx, args)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1212, in parse_args
    rest = Command.parse_args(self, ctx, args)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1048, in parse_args
    value, args = param.handle_parse_result(ctx, opts, args)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1630, in handle_parse_result
    value = invoke_param_callback(self.callback, ctx, self, value)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 123, in invoke_param_callback
    return callback(ctx, param, value)
  File "/venv/src/ckan/ckan/cli/cli.py", line 125, in _init_ckan_config
    _add_ctx_object(ctx, value)
  File "/venv/src/ckan/ckan/cli/cli.py", line 134, in _add_ctx_object
    ctx.obj = CtxObject(path)
  File "/venv/src/ckan/ckan/cli/cli.py", line 56, in __init__
    self.app = make_app(self.config)
  File "/venv/src/ckan/ckan/config/middleware/__init__.py", line 56, in make_app
    load_environment(conf)
  File "/venv/src/ckan/ckan/config/environment.py", line 123, in load_environment
    p.load_all()
  File "/venv/src/ckan/ckan/plugins/core.py", line 165, in load_all
    load(*plugins)
  File "/venv/src/ckan/ckan/plugins/core.py", line 179, in load
    service = _get_service(plugin)
  File "/venv/src/ckan/ckan/plugins/core.py", line 281, in _get_service
    return plugin.load()(name=plugin_name)
  File "/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2443, in load
    return self.resolve()
  File "/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2449, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/venv/src/ckanext-harvest/ckanext/harvest/harvesters/__init__.py", line 1, in <module>
    from ckanext.harvest.harvesters.ckanharvester import CKANHarvester
  File "/venv/src/ckanext-harvest/ckanext/harvest/harvesters/ckanharvester.py", line 16, in <module>
    from .base import HarvesterBase
  File "/venv/src/ckanext-harvest/ckanext/harvest/harvesters/base.py", line 26, in <module>
    from ckanext.harvest.logic.schema import unicode_safe
  File "/venv/src/ckanext-harvest/ckanext/harvest/logic/schema.py", line 13, in <module>
    from ckantoolkit import unicode_safe
  File "<frozen importlib._bootstrap>", line 1040, in _handle_fromlist
  File "/venv/lib/python3.8/site-packages/ckantoolkit/__init__.py", line 19, in __getattr__
    value = getattr(tk, name)
  File "/venv/src/ckan/ckan/plugins/toolkit.py", line 563, in __getattr__
    raise AttributeError('`%s` not found in plugins toolkit' % name)
AttributeError: `__name__` not found in plugins toolkit

I've tracked that down to these two lines:

ckan-pip install -e git+https://github.com/ckan/ckanext-dcat.git@v1.1.3#egg=ckanext-dcat
ckan-pip install -r src/ckanext-dcat/requirements.txt

The second line will remove ckantoolkit@0.0.7 and install ckantoolkit@0.0.3, which prevents ckan from starting. Installing ckanext-dcat from the main branch works fine, as it requires ckantoolkit@0.0.7.

It took me some time to track down this bug, so I wanted to share it. Furthermore I'd like to ask the plugin developers to indicate working ckan and harvester versions somewhere (like within the changelog document) or to file a new tag/release, which works for the current ckan version. Installing a plugin from the main branch is kinda risky, as it might break the next time I'll do so, which is even more important for reproducable builds.

[seitenbau-govdata]

The reason is that the latest commits that fixes the issue is only on master. The latest release/tag v1.1.3 of ckanext-dcat is not compatible with the latest official release/tag v1.4.0 of ckanext-harvest. Until a new version is released you must use the current master.

[amercader]

Sorry @rmeissn I was waiting to see if there were more changes needed for some new CKAN 2.10 related changes to release a new ckanext-dcat version, but I'll do that soon

[seitenbau-govdata]

@rmeissn Version 1.2.0 was released a few days ago. This version should work together with the ckanext-harvest 1.4.0. If it is not work, please reopen the issue.