Unable to perform discovery on chaostoolkit-azure

[zhuweiji]

Under the Chaos Toolkit CLI, the discover command is exposed.

Running the discover command for this package results in an error.

$ chaos discover --no-install chaostoolkit-azure
Traceback (most recent call last):
  File "...\python3.10\latest\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "...\python3.10\latest\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "...\venv\Scripts\chaos.exe\__main__.py", line 7, in <module>
  File "...\venv\lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "...\venv\lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "...\venv\lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "...\venv\lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "...\venv\lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "...\venv\lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "...\venv\lib\site-packages\chaostoolkit\commands\discover.py", line 46, in discover
    discovery = disco(
  File "...\venv\lib\site-packages\chaoslib\discovery\discover.py", line 43, in discover
    package = load_package(package_name)
  File "...\venv\lib\site-packages\chaoslib\discovery\package.py", line 51, in load_package
    name = get_importname_from_package(package_name)
  File "...\venv\lib\site-packages\chaoslib\discovery\package.py", line 117, in get_importname_from_package
    if len(packages) > 1:
TypeError: object of type 'NoneType' has no len()

In comparison, running this command for another chaos-toolkit extension such as the chaostoolkit-kubernetes extension shows the following result.

$ chaos discover chaostoolkit-kubernetes
[2024-07-31 18:12:32 INFO] Attempting to download and install package 'chaostoolkit-kubernetes'
[2024-07-31 18:12:58 INFO] Package downloaded and installed in current environment
[2024-07-31 18:12:59 INFO] Discovering capabilities from chaostoolkit-kubernetes
[2024-07-31 18:12:59 INFO] Searching for actions in chaosk8s.actions
[2024-07-31 18:12:59 INFO] Searching for probes in chaosk8s.probes
[2024-07-31 18:13:00 INFO] Searching for actions in chaosk8s.deployment.actions
[2024-07-31 18:13:00 INFO] Searching for actions in chaosk8s.deployment.probes
[2024-07-31 18:13:00 INFO] Searching for actions in chaosk8s.node.actions
[2024-07-31 18:13:00 INFO] Searching for actions in chaosk8s.node.probes
[2024-07-31 18:13:00 INFO] Searching for actions in chaosk8s.pod.actions
[2024-07-31 18:13:00 INFO] Searching for probes in chaosk8s.pod.probes
[2024-07-31 18:13:00 INFO] Searching for actions in chaosk8s.replicaset.actions
[2024-07-31 18:13:00 INFO] Searching for actions in chaosk8s.service.actions
[2024-07-31 18:13:00 INFO] Searching for actions in chaosk8s.service.probes
[2024-07-31 18:13:00 INFO] Searching for actions in chaosk8s.statefulset.actions
[2024-07-31 18:13:00 INFO] Searching for probes in chaosk8s.statefulset.probes
[2024-07-31 18:13:00 INFO] Searching for actions in chaosk8s.crd.actions
[2024-07-31 18:13:00 INFO] Searching for probes in chaosk8s.crd.probes
[2024-07-31 18:13:00 INFO] Discovery outcome saved in ./discovery.json

Version:

$ chaos info core
NAME                VERSION
CLI                 1.19.0
Core library        1.42.1

$ chaos info extensions
NAME                                    VERSION   LICENSE                       DESCRIPTION
chaostoolkit-azure                      0.17.0    Apache-2.0                    Microsoft Azure

[zhuweiji]

This seems to be because top_level.txt is not included with chaostoolkit_azure-0.17.0.dist-info. This top_level.txt is what is used to resolve top-level package names in chaoslib.discovery.package.PathDistribution.

chaostoolkit_kubernetes-0.28.0.dist-info comes with a top_level.txt.

When building the two projects locally with pdm, top_level.txt is missing in chaostoolkit-azure. It's missing in the 0.17 release on github as well. The pyproject.toml of both projects look identical with regards to the build step and the folder structures are similar, so not sure why top_level.txt is not being generated.

[zhuweiji]

@Lawouach any ideas?

[Lawouach]

Hi @zhuweiji sorry for the lag. I was away quite a bit the last few weeks. I'll look into it tomorrow.

[zhuweiji]

@Lawouach, no worries! I'm suspecting that it might be related to python versions, so I'm trying to build the package with various minor and patch versions of python using tox and pyenv.

[Lawouach]

Hey @zhuweiji again I apologise for my slow responses this summer.

I have reproduced the issue and I believe it doesn't come from this package but from the fact that when we install packages, the wheels distributions do not contain a top_level.txt metadat files any longer. So I had to change the way we discover package name from distributions in chaostoolkit-lib. https://github.com/chaostoolkit/chaostoolkit-lib/commit/9e0d067e3485021c660ec9a38184d77faa9ef1eb

I'm about to release chaostoolkit-lib 1.44.0 and I hope this should fix it.

[Lawouach]

Alright, if you update chaostoolkit-lib the discovery should now work.

$ pip install -U chaostoolkit-lib

$ chaos discover --no-install chaostoolkit-azure
[2024-08-25 11:12:03 INFO] Discovering capabilities from chaostoolkit-azure
[2024-08-25 11:12:03 WARNING] 'chaosazure.webapp.probes' does not expose the __all__ attribute. It is required to determine what functions are actually exported as activities.
[2024-08-25 11:12:03 INFO] Discovery outcome saved in ./discovery.json

[zhuweiji]

Hi Sylvain, thanks so much for the fix. I was looking into it for awhile too but I couldn't figure out how to navigate the importlib_metadata API. I'll mark the issue as closed.