Compatibility

[ChristianF88]

Hi @mutantsan and @mutantsanya,

your ckanext looks promising. I am trying to get it working with my ckan 2.9.11 instance.

Unfortunately I keep on getting errors.

1) NoModuleError for ckan.types caused by these imports for type hints https://github.com/mutantsan/ckanext-unfold/blob/36ce00933388d6dd4dd128c05d8f91a614c31167/ckanext/unfold/plugin.py#L7 and https://github.com/mutantsan/ckanext-unfold/blob/36ce00933388d6dd4dd128c05d8f91a614c31167/ckanext/unfold/logic/validators.py#L4

2) If I remove the type hints, the problem becomes:

Traceback (most recent call last):
  File "/usr/bin/ckan", line 8, in <module>
    sys.exit(ckan())
  File "/usr/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 781, in main
    with self.make_context(prog_name, args, **extra) as ctx:
  File "/usr/lib/python3.9/site-packages/click/core.py", line 700, in make_context
    self.parse_args(ctx, args)
  File "/srv/app/src/ckan/ckan/cli/cli.py", line 116, in parse_args
    result = super(ExtendableGroup, self).parse_args(ctx, args)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 1212, in parse_args
    rest = Command.parse_args(self, ctx, args)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 1048, in parse_args
    value, args = param.handle_parse_result(ctx, opts, args)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 1630, in handle_parse_result
    value = invoke_param_callback(self.callback, ctx, self, value)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 123, in invoke_param_callback
    return callback(ctx, param, value)
  File "/srv/app/src/ckan/ckan/cli/cli.py", line 126, in _init_ckan_config
    _add_ctx_object(ctx, value)
  File "/srv/app/src/ckan/ckan/cli/cli.py", line 135, in _add_ctx_object
    ctx.obj = CtxObject(path)
  File "/srv/app/src/ckan/ckan/cli/cli.py", line 57, in __init__
    self.app = make_app(self.config)
  File "/srv/app/src/ckan/ckan/config/middleware/__init__.py", line 56, in make_app
    load_environment(conf)
  File "/srv/app/src/ckan/ckan/config/environment.py", line 123, in load_environment
    p.load_all()
  File "/srv/app/src/ckan/ckan/plugins/core.py", line 165, in load_all
    load(*plugins)
  File "/srv/app/src/ckan/ckan/plugins/core.py", line 179, in load
    service = _get_service(plugin)
  File "/srv/app/src/ckan/ckan/plugins/core.py", line 281, in _get_service
    return plugin.load()(name=plugin_name)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2443, in load
    return self.resolve()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2449, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/srv/app/src/ckanext-unfold/ckanext/unfold/plugin.py", line 10, in <module>
    from ckanext.unfold.logic.schema import get_preview_schema
  File "/srv/app/src/ckanext-unfold/ckanext/unfold/logic/schema.py", line 11, in <module>
    def get_preview_schema(ignore_empty, unicode_safe, url_validator) -> Schema:
  File "/srv/app/src/ckan/ckan/logic/schema.py", line 16, in validator_args
    args = inspect.getargspec(fn).args
  File "/usr/lib/python3.9/inspect.py", line 1122, in getargspec
    raise ValueError("Function has keyword-only parameters or annotations"
ValueError: Function has keyword-only parameters or annotations, use inspect.signature() API which can support them

Do you have any advice on how to get the extension working?

Cheers!

[mutantsan]

Hello, @ChristianF88! I'm not planning to implement the support for the ckan-2.9.11 version, but you can fork the repo and do it on your own.

1. You'll have to remove type hinting that relies on ckan.types module and replace it with dict[str, Any],

2. As for the second error, I suggest you try to alter the action like this:

   @tk.side_effect_free
   def get_archive_structure(context, data_dict):
   data_dict, errors = tk.navl_validate(
       data_dict,
       {
           "id": [
               tk.get_validator("not_empty"),
               tk.get_validator("unicode_safe"),
               tk.get_validator("resource_id_exists"),
           ],
           "view_id": [
               tk.get_validator("not_empty"),
               tk.get_validator("unicode_safe"),
               tk.get_validator("resource_view_id_exists"),
           ],
       },
   )
   
   if errors:
       raise tk.ValidationError(errors)
   
   resource = tk.get_action("resource_show")(context, {"id": data_dict["id"]})
   resource_view = tk.get_action("resource_view_show")(
       context, {"id": data_dict["view_id"]}
   )
   
   try:
       return [
           n.model_dump() if isinstance(n, unf_types.Node) else n
           for n in unf_utils.get_archive_tree(resource, resource_view)
       ]
   except unf_exception.UnfoldError as e:
       return {"error": str(e)}

We remove the validate decorator and move the schema validation inside the function body.

[ChristianF88]

Thanks @mutantsan! I'll definitely have a look at it. Which ckan version does the code work with? That would be helpful to know, then I can also have a look at the official Changelog.

[mutantsan]

It was written for CKAN 2.10+. I had to specify it in the readme, but forgot to do it 😞

[mutantsan]

Closing this issue. Feel free to reopen it if you have other questions.

[ChristianF88]

Hi @mutantsan,

I spent some time today on making the code ckan 2.9 compatible. I got it running without errors, but the extension doesn't work properly. When I create a view the extension gets stuck here:

At this point I will not spend more time on this, once we update to CKAN 2.10, I'll be back ;)

Here's a list of the necessary changes, in case someone else wants to tackle this:

1) Remove ckan type hints (plugin.py, validator.py) 2) Add validator_args (without type hints) to schema.py:

from functools import wraps
from ckan.logic import get_validator
import inspect

# Copied from CKAN 2.10, removed type hints
def validator_args(fn):
    """Collect validator names from argument names and pass them to wrapped
    function.

    Example::

        @validator_args
        def schema_function(not_empty, ignore):
            return not_empty, ignore

        ne, ig = schema_function()
        assert ne is get_validator("not_empty")
        assert ig is get_validator("ignore")
    """

    args = inspect.signature(fn).parameters

    @wraps(fn)
    def wrapper():
        kwargs = {
            arg: get_validator(arg)
            for arg in args}
        return fn(**kwargs)

    return wrapper
...

3) Remove blanket decorators (@tk.blanket.actions, @tk.blanket.validators) from plugin.py, as they were introduced in CKAN 2.10.