search

galaxyproject / planemo

Command-line utilities to assist in developing Galaxy and Common Workflow Language artifacts - including tools, workflows, and training materials.
https://planemo.readthedocs.io/
MIT License
86 stars 85 forks source link

planemo lint fails with lxml version 5.0.0 #1421

Closed patrick-austin closed 6 months ago

patrick-austin commented 6 months ago

Running planemo lint / planemo shed_lint locally or by CI fails (when reaching the xsd section) with:

Traceback (most recent call last):
  File "/home/kez92691/.venv_tmp/bin/planemo", line 8, in <module>
    sys.exit(planemo())
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/planemo/cli.py", line 102, in handle_blended_options
    return f(*args, **kwds)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/planemo/commands/cmd_shed_lint.py", line 62, in cli
    exit_code = shed.for_each_repository(ctx, lint, paths, **kwds)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/planemo/shed/__init__.py", line 762, in for_each_repository
    ret_codes.append(function(realized_repository))
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/planemo/commands/cmd_shed_lint.py", line 59, in lint
    return shed_lint.lint_repository(ctx, realized_repository, **kwds)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/planemo/shed_lint.py", line 111, in lint_repository
    tools_failed = lint_repository_tools(ctx, realized_repository, lint_ctx, lint_args)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/planemo/shed_lint.py", line 129, in lint_repository_tools
    lint_tool_source_with(lint_ctx, tool_source, extra_modules=lint_args["extra_modules"])
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/galaxy/tool_util/lint.py", line 341, in lint_tool_source_with
    lint_context.lint(name, value, tool_xml)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/galaxy/tool_util/lint.py", line 184, in lint
    lint_func(lint_target, self)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/planemo/linters/xsd.py", line 17, in lint_tool_xsd
    planemo.lint.lint_xsd(lint_ctx, TOOL_XSD, tf.name)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/planemo/lint.py", line 93, in lint_xsd
    validation_result = validator.validate(schema_path, path)
  File "/home/kez92691/.venv_tmp/lib/python3.8/site-packages/planemo/xml/validation.py", line 50, in validate
    xsd = etree.XMLSchema(xsd_doc)
  File "src/lxml/xmlschema.pxi", line 89, in lxml.etree.XMLSchema.__init__
lxml.etree.XMLSchemaParseError: complex type 'Output': The content model is not determinist., line 5329

When the latest version of lxml (5.0.0) is installed (which the CI does automatically). Installing an older version (tested with 4.9.4 locally) does not cause the same error.

Perhaps this is an underlying problem with lxml that will be fixed in time, but if not then I would've thought the version just needs to be pinned to fix this: https://github.com/galaxyproject/planemo/blob/f6ce25ca689e7dc4f03e67d7b40fdd4d98e204b2/requirements.txt#L12

bgruening commented 6 months ago

@patrick-austin the newer version of lxml is more strict and causes those errors. You can avoid that by pip install 'lxml<5'. Marius has fixed that already but we need to release new Galaxy packages to pypi. Everyone is on vacation so this can take a few days.

patrick-austin commented 6 months ago

Thanks @bgruening , good to know there's already awareness/fix of this - I couldn't see an issue about it here so thought it better to report it than not. I'm using lxml==4.9.4 locally, but the CI will still fail. Though having said that I suppose adding a step to GHA pinning lxml would be a workaround if needed. Probably easiest to just wait for the release.

mvdbeek commented 6 months ago

Packages are released, this should work now.