Backend loading is dependent on setuptools

[matthewfeickert]

If recast-atlas is installed in an environment without setuptools

uv venv
. .venv/bin/activate
uv pip install --upgrade pip wheel
uv --no-cache pip install 'recast-atlas[local]>=0.4.2' 'yadage<0.21.0' coolname
uv --no-cache pip install --upgrade 'yadage<0.21.0'  # have to do this twice with uv v0.2.6 to get yadage bound respected
uv pip list | grep setuptools  # nothing, as not in the environment as expected

then running with a non-docker backend

PACKTIVITY_DOCKER_CMD_MOD="-u root" recast run examples/rome --backend local --tag "local-$(coolname 2)"

will fail with

2024-06-17 12:45:53,511 | recastatlas.subcomma |  ERROR | caught exception
Traceback (most recent call last):
  File "/home/feickert/.pyenv/versions/3.12.1/envs/juhofer-debug/lib/python3.12/site-packages/recastatlas/subcommands/run.py", line 59, in run
    run_sync(name, spec, backend=backend)
  File "/home/feickert/.pyenv/versions/3.12.1/envs/juhofer-debug/lib/python3.12/site-packages/recastatlas/backends/__init__.py", line 78, in run_sync
    BACKENDS[backend].run_workflow(name, spec)
    ~~~~~~~~^^^^^^^^^
KeyError: 'local'
Error: Workflow failed

This is resolved if setuptools is installed

uv pip install setuptools

but recast-atlas should have no dependence on setuptools at all.

https://github.com/recast-hep/recast-atlas/blob/11af091e2d5651b33ba234e55e48cd322756597a/pyproject.toml#L1-L6

---

https://github.com/recast-hep/recast-atlas/blob/11af091e2d5651b33ba234e55e48cd322756597a/src/recastatlas/subcommands/run.py#L58-L64

https://github.com/recast-hep/recast-atlas/blob/11af091e2d5651b33ba234e55e48cd322756597a/src/recastatlas/backends/__init__.py#L76-L78

[matthewfeickert]

Setting a breakpoint just before

BACKENDS[backend].run_workflow(name, spec)

in

https://github.com/recast-hep/recast-atlas/blob/11af091e2d5651b33ba234e55e48cd322756597a/src/recastatlas/backends/__init__.py#L76-L78

results in

$ PACKTIVITY_DOCKER_CMD_MOD="-u root" recast run examples/rome --backend local --tag "local-$(coolname 2)"
> /home/feickert/.pyenv/versions/3.12.1/envs/juhofer-debug/lib/python3.12/site-packages/recastatlas/backends/__init__.py(79)run_sync()
-> BACKENDS[backend].run_workflow(name, spec)
(Pdb) backend
'local'
(Pdb) BACKENDS
{'kubernetes': <recastatlas.backends.kubernetes.KubernetesBackend object at 0x7f0539fe5250>, 'docker': <recastatlas.backends.docker.DockerBackend object at 0x7f053abe6330>}
(Pdb) from recastatlas.backends.local import LocalBackend
*** ModuleNotFoundError: No module named 'pkg_resources'
(Pdb) 

So even though pkg_resources isn't used in the code base directly, it is getting used somewhere.

[matthewfeickert]

The problem is that

https://github.com/recast-hep/recast-atlas/blob/11af091e2d5651b33ba234e55e48cd322756597a/src/recastatlas/backends/__init__.py#L31-L36

calls

https://github.com/recast-hep/recast-atlas/blob/11af091e2d5651b33ba234e55e48cd322756597a/src/recastatlas/backends/local.py#L7

and yadage depends on packtivity which depends on checksumdir which uses pkg_resources.

>>> from yadage.steering_api import run_workflow
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/feickert/.pyenv/versions/juhofer-debug/lib/python3.12/site-packages/yadage/steering_api.py", line 8, in <module>
    from .steering_object import YadageSteering
  File "/home/feickert/.pyenv/versions/juhofer-debug/lib/python3.12/site-packages/yadage/steering_object.py", line 8, in <module>
    from .wflowstate import load_model_fromstring
  File "/home/feickert/.pyenv/versions/juhofer-debug/lib/python3.12/site-packages/yadage/wflowstate.py", line 4, in <module>
    from .wflow import YadageWorkflow
  File "/home/feickert/.pyenv/versions/juhofer-debug/lib/python3.12/site-packages/yadage/wflow.py", line 4, in <module>
    from .stages import JsonStage, OffsetStage
  File "/home/feickert/.pyenv/versions/juhofer-debug/lib/python3.12/site-packages/yadage/stages.py", line 5, in <module>
    from .state_providers import load_provider
  File "/home/feickert/.pyenv/versions/juhofer-debug/lib/python3.12/site-packages/yadage/state_providers/__init__.py", line 4, in <module>
    from .localposix import LocalFSProvider
  File "/home/feickert/.pyenv/versions/juhofer-debug/lib/python3.12/site-packages/yadage/state_providers/localposix.py", line 6, in <module>
    from packtivity.statecontexts import load_state
  File "/home/feickert/.pyenv/versions/juhofer-debug/lib/python3.12/site-packages/packtivity/statecontexts/__init__.py", line 4, in <module>
    from .posixfs_context import LocalFSState
  File "/home/feickert/.pyenv/versions/juhofer-debug/lib/python3.12/site-packages/packtivity/statecontexts/posixfs_context.py", line 6, in <module>
    import checksumdir
  File "/home/feickert/.pyenv/versions/juhofer-debug/lib/python3.12/site-packages/checksumdir/__init__.py", line 17, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

[matthewfeickert]

checksumdir looks unsupported given https://github.com/to-mc/checksumdir/issues/26, so I think the solution is to move packtivity off of it for an alternative.

[matthewfeickert]

Resolved by the release of packtivity v0.17.0.