test_run_hello_world test fails on Python 3.9

[matthewfeickert]

Trying to run the tests with Python 3.9 in the CI fails with the following

$ python -m pytest tests

============================= test session starts ==============================
platform linux -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /home/runner/work/recast-atlas/recast-atlas
collected 3 items

tests/test_cli.py .F.                                                    [100%]

=================================== FAILURES ===================================
_____________________________ test_run_hello_world _____________________________

tmpdir = local('/tmp/pytest-of-runner/pytest-0/test_run_hello_world0')

    def test_run_hello_world(tmpdir):
        with tmpdir.as_cwd():
            runner = CliRunner()
            test = runner.invoke(
                run, ['testing/busyboxtest', '--backend', 'local', '--tag', 'hello']
            )
>           assert test.exit_code == 0
E           assert 1 == 0
E            +  where 1 = <Result SystemExit(1)>.exit_code

tests/test_cli.py:20: AssertionError
------------------------------ Captured log call -------------------------------
ERROR    recastatlas.subcommands.run:run.py:58 caught exception
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.9.7/x64/lib/python3.9/site-packages/recastatlas/subcommands/run.py", line 56, in run
    run_sync(name, spec, backend=backend)
  File "/opt/hostedtoolcache/Python/3.9.7/x64/lib/python3.9/site-packages/recastatlas/backends/__init__.py", line 76, in run_sync
    BACKENDS[backend].run_workflow(name, spec)
KeyError: 'local'
=========================== short test summary info ============================
FAILED tests/test_cli.py::test_run_hello_world - assert 1 == 0
========================= 1 failed, 2 passed in 0.30s ==========================
Error: Process completed with exit code 1.

I'm unclear on why Python 3.8 passes but Python 3.9 fails.

[matthewfeickert]

Localizing this a bit:

• Python 3.8

  $ python -c 'from yadage.steering_api import run_workflow'  # no output as import works

• Python 3.9

  $ python -c 'from yadage.steering_api import run_workflow'
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/feickert/.pyenv/versions/recast-atlas-39/lib/python3.9/site-packages/yadage/steering_api.py", line 8, in <module>
  from .steering_object import YadageSteering
  File "/home/feickert/.pyenv/versions/recast-atlas-39/lib/python3.9/site-packages/yadage/steering_object.py", line 5, in <module>
  import adage
  File "/home/feickert/.pyenv/versions/recast-atlas-39/lib/python3.9/site-packages/adage/__init__.py", line 6, in <module>
  from adage.adageobject import adageobject
  File "/home/feickert/.pyenv/versions/recast-atlas-39/lib/python3.9/site-packages/adage/adageobject.py", line 1, in <module>
  import adage.graph
  File "/home/feickert/.pyenv/versions/recast-atlas-39/lib/python3.9/site-packages/adage/graph.py", line 1, in <module>
  import networkx as nx
  File "/home/feickert/.pyenv/versions/recast-atlas-39/lib/python3.9/site-packages/networkx/__init__.py", line 84, in <module>
  import networkx.generators
  File "/home/feickert/.pyenv/versions/recast-atlas-39/lib/python3.9/site-packages/networkx/generators/__init__.py", line 5, in <module>
  from networkx.generators.classic import *
  File "/home/feickert/.pyenv/versions/recast-atlas-39/lib/python3.9/site-packages/networkx/generators/classic.py", line 21, in <module>
  from networkx.algorithms.bipartite.generators import complete_bipartite_graph
  File "/home/feickert/.pyenv/versions/recast-atlas-39/lib/python3.9/site-packages/networkx/algorithms/__init__.py", line 12, in <module>
  from networkx.algorithms.dag import *
  File "/home/feickert/.pyenv/versions/recast-atlas-39/lib/python3.9/site-packages/networkx/algorithms/dag.py", line 2, in <module>
  from fractions import gcd
  ImportError: cannot import name 'gcd' from 'fractions' (/home/feickert/.pyenv/versions/3.9.6/lib/python3.9/fractions.py)

where

$ pip show networkx
Name: networkx
Version: 1.11
Summary: Python package for creating and manipulating graphs and networks
Home-page: http://networkx.github.io/
Author: NetworkX Developers
Author-email: networkx-discuss@googlegroups.com
License: BSD
Location: /home/feickert/.pyenv/versions/3.9.6/envs/recast-atlas-39/lib/python3.9/site-packages
Requires: decorator
Required-by: adage

As described in https://github.com/smicallef/spiderfoot/issues/1124, this is due to networkx<2.5 using a deprecated function. So as adage is apparently pinning this with

    install_requires = [
        'networkx==1.11'
    ],

Luckily, yadage requires adage

$ pip show adage
Name: adage
Version: 0.10.1
Summary: running dynamic DAG workflows
Home-page: UNKNOWN
Author: Lukas Heinrich
Author-email: lukas.heinrich@gmail.com
License: UNKNOWN
Location: /home/feickert/.pyenv/versions/3.9.6/envs/recast-atlas-39/lib/python3.9/site-packages
Requires: networkx
Required-by: yadage

but doesn't have it pinned.

So the minimum fix for this should be:

1. Upgrade networkx dependency in adage to be networkx>=2.5
2. Fix anything that breaks with that
3. Make new release of adage

[matthewfeickert]

Resolved with release of recast-atlas v0.1.9 given the release of adage v0.10.2.