DeprecationWarning for declare_namespace

[mdickinson]

I'm seeing the following DeprecationWarning repeatedly in a local test run (Python 3.10, setuptools 67.6.0):

/private/var/folders/07/jbbjv8b53bs5y9xyjdyn1zgw0000gn/T/tmp9y_u_y0o/acme.bar-0.1a1-py3.10.egg/acme/bar/__init__.py:11: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('acme.bar')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages

[mdickinson]

Steps to reproduce, here using Python 3.11 (on the current main branch, commit 886381e1f7b20e658752e27c5b77c298f300b444; macOS + zsh):

% python3.11 -m venv --clear ~/.venvs/envisage
% source ~/.venvs/envisage/bin/activate
% python -m pip install --upgrade pip setuptools
% python -m pip install .
% python -m unittest -v

The relevant portion of the test output looks like this:

test_find_plugins_in_eggs_on_the_plugin_path (envisage.tests.test_egg_basket_plugin_manager.EggBasketPluginManagerTestCase.test_find_plugins_in_eggs_on_the_plugin_path) ... /private/var/folders/07/jbbjv8b53bs5y9xyjdyn1zgw0000gn/T/tmp_tqlmf24/acme.bar-0.1a1-py3.11.egg/acme/__init__.py:11: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('acme')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
/private/var/folders/07/jbbjv8b53bs5y9xyjdyn1zgw0000gn/T/tmp_tqlmf24/acme.foo-0.1a1-py3.11.egg/acme/foo/__init__.py:11: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('acme.foo')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
/Users/mdickinson/.venvs/envisage/lib/python3.11/site-packages/pkg_resources/__init__.py:2349: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('acme')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(parent)
/private/var/folders/07/jbbjv8b53bs5y9xyjdyn1zgw0000gn/T/tmp_tqlmf24/acme.bar-0.1a1-py3.11.egg/acme/bar/__init__.py:11: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('acme.bar')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
/private/var/folders/07/jbbjv8b53bs5y9xyjdyn1zgw0000gn/T/tmp_tqlmf24/acme.baz-0.1a1-py3.11.egg/acme/baz/__init__.py:11: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('acme.baz')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
ok

[mdickinson]

Looks like the most recent version of setuptools is trying to push us away from using old-style namespace packages. I've converted our test packages to new-style implicit namespace packages successfully, but the EggBasketPluginManager doesn't like them much. Investigating.

[mdickinson]

Looking at e.g. test_ignore_broken_plugins_loads_good_plugins, we're building eggs and extracting their entry points without any issue. The problem comes when we use the EntryPoint.load() method to load the entry point - at that point we get a ModuleNotFoundError("No module named 'acme'"). It's not clear to me whether this is because setuptools simply doesn't support doing this for new-style namespace packages, or whether we're doing something wrong.

[mdickinson]

Okay, I think it's because setuptools genuinely doesn't support this. For the EggBasketPluginManager, we're relying on putting paths to eggs on sys.path. That's a very setuptools-y thing to do, and it seems that if we have an egg on sys.path and that egg contains a new-style namespace package, then a simple import acme doesn't pick up the acme namespace that's nested in the egg (which isn't entirely surprising).

This is more motivation to deprecate EggBasketPluginManager and replace it with something based directly on importlib.metadata entry points.

[mdickinson]

Another possible way forward here is to get rid of the namespace packages altogether - it's not at all clear to me why acme.foo, acme.bar, etc. are set up as namespace packages in the first place.