search

nornir-automation / nornir

Pluggable multi-threaded framework with inventory management to help operate collections of devices
https://nornir.readthedocs.io/
Apache License 2.0
1.37k stars 233 forks source link

remove deprecated mypy_extensions and typing_extensions deps #863

Closed nemith closed 11 months ago

nemith commented 11 months ago

mypy_extensions and typing_extensions were decalred only as dev depenendencies. This means in a clean enviroment (like a fresh venv) nornir would fail to load.

Turns out the use for mypy_extensions was deprecated so we can remove that also Protocol is part of the base typing package so we can get rid of the typing_extensions package as well fixing both.

See: https://mypy.readthedocs.io/en/stable/additional_features.html?highlight=mypy_extensions#extended-callable-types

Fixes: #861

Testing

Before (on master)

$ git switch main
Switched to branch 'main'
Your branch is up to date with 'upstream/main'.
$ rm -rf /tmp/venv && python3 -m venv /tmp/venv && source /tmp/venv/bin/activate
$ pip install .
Processing /Volumes/Code/nornir
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting ruamel.yaml>=0.17 (from nornir==3.4.0)
  Obtaining dependency information for ruamel.yaml>=0.17 from https://files.pythonhosted.org/packages/d9/0e/2a05efa11ea33513fbdf4a2e2576fe94fd8fa5ad226dbb9c660886390974/ruamel.yaml-0.17.32-py3-none-any.whl.metadata
  Using cached ruamel.yaml-0.17.32-py3-none-any.whl.metadata (17 kB)
Collecting ruamel.yaml.clib>=0.2.7 (from ruamel.yaml>=0.17->nornir==3.4.0)
  Obtaining dependency information for ruamel.yaml.clib>=0.2.7 from https://files.pythonhosted.org/packages/f5/23/b8ff333e40fa194678b01b66c1aced9dc5ecbc16a043b0d09beb6a37377c/ruamel.yaml.clib-0.2.7-cp311-cp311-macosx_13_0_arm64.whl.metadata
  Using cached ruamel.yaml.clib-0.2.7-cp311-cp311-macosx_13_0_arm64.whl.metadata (2.2 kB)
Using cached ruamel.yaml-0.17.32-py3-none-any.whl (112 kB)
Using cached ruamel.yaml.clib-0.2.7-cp311-cp311-macosx_13_0_arm64.whl (125 kB)
Building wheels for collected packages: nornir
  Building wheel for nornir (pyproject.toml) ... done
  Created wheel for nornir: filename=nornir-3.4.0-py3-none-any.whl size=30570 sha256=cff7a6cfbb4ba5d4d0427f78ae65d8275e2e30412d5e68ec54a210cab4371236
  Stored in directory: /private/var/folders/xv/k7lms_fn08972kd179h5j1zr0000gp/T/pip-ephem-wheel-cache-vl5mhvvm/wheels/97/89/d9/720bf7ca5f7e0b4e0392b119fd063d14f3782a0d72f9116fa3
Successfully built nornir
Installing collected packages: ruamel.yaml.clib, ruamel.yaml, nornir
Successfully installed nornir-3.4.0 ruamel.yaml-0.17.32 ruamel.yaml.clib-0.2.7
$ pip freeze
nornir @ file:///Volumes/Code/nornir
ruamel.yaml==0.17.32
ruamel.yaml.clib==0.2.7
$ python3
Python 3.11.5 (main, Aug 24 2023, 15:09:45) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from nornir.init_nornir import InitNornir
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Volumes/Code/nornir/nornir/__init__.py", line 3, in <module>
    from nornir.init_nornir import InitNornir
  File "/Volumes/Code/nornir/nornir/init_nornir.py", line 3, in <module>
    from nornir.core import Nornir
  File "/Volumes/Code/nornir/nornir/core/__init__.py", line 6, in <module>
    from nornir.core.inventory import Inventory
  File "/Volumes/Code/nornir/nornir/core/inventory.py", line 17, in <module>
    from nornir.core.plugins.connections import (
  File "/Volumes/Code/nornir/nornir/core/plugins/connections.py", line 6, in <module>
    from typing_extensions import Protocol
ModuleNotFoundError: No module named 'typing_extensions'
>>> print("sad elephant")
sad elephant
>>>

After my fix

 rm -rf /tmp/venv && python3 -m venv /tmp/venv && source /tmp/venv/bin/activate
$ pip install .
Processing /Volumes/Code/nornir
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting ruamel.yaml>=0.17 (from nornir==3.4.0)
  Obtaining dependency information for ruamel.yaml>=0.17 from https://files.pythonhosted.org/packages/d9/0e/2a05efa11ea33513fbdf4a2e2576fe94fd8fa5ad226dbb9c660886390974/ruamel.yaml-0.17.32-py3-none-any.whl.metadata
  Using cached ruamel.yaml-0.17.32-py3-none-any.whl.metadata (17 kB)
Collecting typing_extensions<5.0.0,>=4.7.1 (from nornir==3.4.0)
  Obtaining dependency information for typing_extensions<5.0.0,>=4.7.1 from https://files.pythonhosted.org/packages/24/21/7d397a4b7934ff4028987914ac1044d3b7d52712f30e2ac7a2ae5bc86dd0/typing_extensions-4.8.0-py3-none-any.whl.metadata
  Using cached typing_extensions-4.8.0-py3-none-any.whl.metadata (3.0 kB)
Collecting ruamel.yaml.clib>=0.2.7 (from ruamel.yaml>=0.17->nornir==3.4.0)
  Obtaining dependency information for ruamel.yaml.clib>=0.2.7 from https://files.pythonhosted.org/packages/f5/23/b8ff333e40fa194678b01b66c1aced9dc5ecbc16a043b0d09beb6a37377c/ruamel.yaml.clib-0.2.7-cp311-cp311-macosx_13_0_arm64.whl.metadata
  Using cached ruamel.yaml.clib-0.2.7-cp311-cp311-macosx_13_0_arm64.whl.metadata (2.2 kB)
Using cached ruamel.yaml-0.17.32-py3-none-any.whl (112 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached ruamel.yaml.clib-0.2.7-cp311-cp311-macosx_13_0_arm64.whl (125 kB)
Building wheels for collected packages: nornir
  Building wheel for nornir (pyproject.toml) ... done
  Created wheel for nornir: filename=nornir-3.4.0-py3-none-any.whl size=30594 sha256=0d390bd2603b0cb2276eab8fcc0b2be8727fa60ddc90980964ac250a82bcd3f6
  Stored in directory: /private/var/folders/xv/k7lms_fn08972kd179h5j1zr0000gp/T/pip-ephem-wheel-cache-zsiapwf0/wheels/97/89/d9/720bf7ca5f7e0b4e0392b119fd063d14f3782a0d72f9116fa3
Successfully built nornir
Installing collected packages: typing_extensions, ruamel.yaml.clib, ruamel.yaml, nornir
Successfully installed nornir-3.4.0 ruamel.yaml-0.17.32 ruamel.yaml.clib-0.2.7 typing_extensions-4.8.0
$ pip freeze
nornir @ file:///Volumes/Code/nornir
ruamel.yaml==0.17.32
ruamel.yaml.clib==0.2.7
typing_extensions==4.8.0
$ python3
Python 3.11.5 (main, Aug 24 2023, 15:09:45) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from nornir.init_nornir import InitNornir
>>> print("awesome possum")
awesome possum
>>> ^D
nemith commented 11 months ago

And from looking at your code, I assume you did not use black for the formatting. The code must be formatted with black to pass the test.

https://github.com/nornir-automation/nornir/blob/325c8ca479b08662a0f963a90d3328a34ef17f2a/Makefile#L17-L19

Just the opposite. My editor is setup to run black and isort and isort is formatting the imports (black doesn't)

nemith commented 11 months ago

It would be great not to add a dependency again. Luckily we removed Python 3.7 support, so we are fine to use typing.Protocol

I can update to use typing.Protocol. I am not ADDING a dependency but FIXING one. Fixed ALL the typing_extension imports as well in the latest update.

ktbyers commented 11 months ago

Fix implemented here: #867

I created a separate issue related to the removal of mypy_extensions.