Invoke commands are not working

[MarkusH]

The contribution guide says, one should run poetry install, poetry shell and then invoke setup. However, the last command fails on Python 3.11

Traceback (most recent call last):
  File "/home/markus/.cache/pypoetry/virtualenvs/kagi-dUIZ9Tgo-py3.11/bin/invoke", line 8, in <module>
    sys.exit(program.run())
             ^^^^^^^^^^^^^
  File "/home/markus/.cache/pypoetry/virtualenvs/kagi-dUIZ9Tgo-py3.11/lib/python3.11/site-packages/invoke/program.py", line 373, in run
    self.parse_collection()
  File "/home/markus/.cache/pypoetry/virtualenvs/kagi-dUIZ9Tgo-py3.11/lib/python3.11/site-packages/invoke/program.py", line 465, in parse_collection
    self.load_collection()
  File "/home/markus/.cache/pypoetry/virtualenvs/kagi-dUIZ9Tgo-py3.11/lib/python3.11/site-packages/invoke/program.py", line 699, in load_collection
    module, parent = loader.load(coll_name)
                     ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markus/.cache/pypoetry/virtualenvs/kagi-dUIZ9Tgo-py3.11/lib/python3.11/site-packages/invoke/loader.py", line 76, in load
    module = imp.load_module(name, fd, path, desc)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/imp.py", line 235, in load_module
    return load_source(name, filename, file)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/imp.py", line 172, in load_source
    module = _load(spec)
             ^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 721, in _load
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/markus/Coding/kagi/tasks.py", line 26, in <module>
    @task
     ^^^^
  File "/home/markus/.cache/pypoetry/virtualenvs/kagi-dUIZ9Tgo-py3.11/lib/python3.11/site-packages/invoke/tasks.py", line 331, in task
    return klass(args[0], **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markus/.cache/pypoetry/virtualenvs/kagi-dUIZ9Tgo-py3.11/lib/python3.11/site-packages/invoke/tasks.py", line 76, in __init__
    self.positional = self.fill_implicit_positionals(positional)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markus/.cache/pypoetry/virtualenvs/kagi-dUIZ9Tgo-py3.11/lib/python3.11/site-packages/invoke/tasks.py", line 167, in fill_implicit_positionals
    args, spec_dict = self.argspec(self.body)
                      ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/markus/.cache/pypoetry/virtualenvs/kagi-dUIZ9Tgo-py3.11/lib/python3.11/site-packages/invoke/tasks.py", line 153, in argspec
    spec = inspect.getargspec(func)
           ^^^^^^^^^^^^^^^^^^
AttributeError: module 'inspect' has no attribute 'getargspec'. Did you mean: 'getargs'?

I'd like to suggest to move away and drop invoke in favor of tools like tox for test matrices as well as a hand-full documented commands, that work independently of the Python version.

The fact, that the tasks.py fails with the above error also means, running pytest manually fails.

[MarkusH]

@justinmayer Here's my proposal for a "milestone", trying to, IMO (yes, highly opinionated), reduce the complexity of tooling:

1. Drop poetry in favor of plain setup.py/pyproject.toml with pip install
2. Use twisted to upload to PyPI from the CI. Essentially leveraging PyPI's Trusted Publisher
3. As part of the release, we could leverage setuptools-scm for retrieve the version number from the git tag
4. Have a requirements/dev.txt and requirements/docs.txt (which we could, if we want, extend with pip-tools/pip-compile
5. Drop pytest and rely on plain old Python unittest
6. Use tox to support test matrices with different Python/Django/webauthn versions

[justinmayer]

I noticed that error yesterday, which is related to an incompatibility between Invoke 1.x and Python 3.11. The easiest resolution is to bump Invoke to 2.0+ in the pyproject file.

[justinmayer]

Thanks for the suggestions. I also have some significantly opinionated thoughts on tooling, so I’ll respond to these ideas as soon as I get back to my desk. 😊

[justinmayer]

Many thanks for the thoughtful suggestions, Markus. Regarding the original genesis for this issue, I see that you have already resolved the problem by upgrading to Invoke 2.x via https://github.com/justinmayer/kagi/commit/4a7f3a7b0fd5c5718f787fbc5c4b5de70097c418. You mentioned that running pytest manually fails, but I can't reproduce that, and at least in theory that should not have had anything to do with the incompatibility between Invoke 1.x and Python 3.11.

Some quick thoughts about the suggestions…

• Poetry is currently used in conjunction with AutoPub to handle the release automation, so in order to retain the benefits provided by AutoPub, before dropping Poetry from this project I'd prefer to add support to AutoPub for alternative tool-chains such as Hatch/Hatchling, PDM, etc. That said, I am not attached to Poetry and am open to switching to alternative tooling that supports pyproject.toml.

• Since AutoPub already uploads Kagi packages to PyPI via CI, I think it would be great to add support to AutoPub for PyPI’s new Trusted Publisher feature.

• I have personally enjoyed the transition from listing dependencies in requirements.txt files to instead specifying them in pyproject.toml, but we can certainly talk about the relative pros & cons of those approaches.

• I am not sure what issues pytest might present, but I haven't encountered any problems with it, and I quite appreciate its modular design and rich plugin ecosystem.

• I normally rely on CI to test the full range of version matrices, running tests locally on a sub-set of all those version combinations. That said, I have no objection to adding tox to the project, provided we also retain the current easy way to run tests locally on a single Python+Django version combination.

In short, I think it might make sense to elaborate regarding the perceived complexity and about what actual, current problems it may pose here.