nornir-automation / nornir

Pluggable multi-threaded framework with inventory management to help operate collections of devices
https://nornir.readthedocs.io/
Apache License 2.0
1.38k stars 234 forks source link

Replace pkg_resources with importlib #731

Closed ubaumann closed 2 years ago

ubaumann commented 2 years ago

I was using nornir in a jupyter notebook and realized the plugin auto register did not work.

The following snippet gives an empty list in the newest jupyter notebook and works fine in REPL.

import pkg_resources
pkg_resources.iter_entry_points("nornir.plugins.connections")

After some investigation I ended up with this snippet working in both environments:

from importlib import metadata
metadata.entry_points()["nornir.plugins.connections"]

Setuptools is recommending using importlib.metadata in the documentation: https://setuptools.pypa.io/en/latest/userguide/entry_point.html#advertising-behavior

If I can change or add anything please let me know

dbarrosop commented 2 years ago

Nice work, just one comment, the docs you linked mentions the new metadata functionality was introduced in 3.8, which I can confirm with docker:

$ docker run --rm -it python:3.8
Python 3.8.12 (default, Oct 16 2021, 10:26:20) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib import metadata
>>> metadata.entry_points()
{'console_scripts': (EntryPoint(name='wheel',...}

But in your PR you set the dependency and the import for <3.10. What am I missing here?

dbarrosop commented 2 years ago

This is the exact line I am referring to;

For a project wishing to solicit entry points, Setuptools recommends the importlib.metadata module (part of stdlib since Python 3.8) or its backport, importlib_metadata.

From https://setuptools.pypa.io/en/latest/userguide/entry_point.html#advertising-behavior

ubaumann commented 2 years ago

Yes I was really confused in the beginning. I had everything running in 3.8 but mypy didn't like me in py3.9 so I looked deeper and saw a fundamental API change between 3.8 and 3.10.

The document calls importlib.metadata "no longer provisional" in 3.10.

https://docs.python.org/3/library/importlib.metadata.html

The API in 3.10 and the backport lib is now the same and works well as I think.

ubaumann commented 2 years ago

In the readme of the backport lib https://pypi.org/project/importlib-metadata/ you can see from version 4.4 it is compatible with Python 3.10

ubaumann commented 2 years ago

3.8

Python 3.8.12 (default, Oct 13 2021, 09:15:35)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib import metadata
>>> type(metadata.entry_points())
<class 'dict'>

3.9

Python 3.9.7 (default, Sep  3 2021, 20:10:26)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib import metadata
>>> type(metadata.entry_points())
<class 'dict'>

3.10

Python 3.10.0 (default, Oct  5 2021, 23:39:58) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib import metadata
>>> type(metadata.entry_points())
<class 'importlib.metadata.SelectableGroups'>

3.9 with pip install importlib-metadata

Python 3.9.7 (default, Sep  3 2021, 20:10:26)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib_metadata as metadata
>>> type(metadata.entry_points())
<class 'importlib_metadata.SelectableGroups'>
dbarrosop commented 2 years ago

I figured there was something, would you mind adding a note in the pyproject.toml for future reference? Otherwise I am sure someone will be doing this same spelunking again.

Thanks!

ubaumann commented 2 years ago

Sure. I added a short note.

Thanks for your feedback

dbarrosop commented 2 years ago

Awesome, thanks to you for the PR!