Support dynamically adding a device_type

[chrisinmtown]

Can Netmiko allow dynamically adding a driver? I need to use a custom authentication scheme for a Cisco router that presents a challenge. Paramiko's paramiko.Transport class provides a auth_interactive method which accepts a callback method to delegate the interaction to custom code. With a custom driver this is (relatively) straightforward, but telling Netmiko about that driver is not. Because this seems to be specific to my organization, it will be very difficult for me to contribute new driver source. Thanks in advance for any pointers!

[ktbyers]

@chrisinmtown Just trying to make sure I understand this properly.

So it just provides you with base64 data and you need to respond with the decoded version of the data plus your password?

[ktbyers]

@chrisinmtown Yeah, can you post what you do in Paramiko and I can potentially see how we can tie that into Netmiko (scrubbed of anything confidential).

[chrisinmtown]

To avoid revealing any internal details, I think I have to leave this very short. Basically the ask is, can Netmiko allow use of my own custom driver that uses Paramiko's auth_interactive method?

This SO answer demonstrates how it's done: https://stackoverflow.com/a/43943658/1630244

[ktbyers]

@chrisinmtown

See Palo Alto Driver for an example of Netmiko using auth_interactive:

https://github.com/ktbyers/netmiko/blob/develop/netmiko/paloalto/paloalto_panos.py#L11

And here:

https://github.com/ktbyers/netmiko/blob/develop/netmiko/paloalto/paloalto_panos.py#L240

[ktbyers]

Let me know if you are able to solve your problem with that or if you are still running into issues.

Regards, Kirk

[chrisinmtown]

Thank you very much for the pointer to code that demonstrates how to override Paramiko's _auth method to perform interactive authentication! Looks like this is a solved problem so I'm closing this issue.

[chrisinmtown]

I'm coming back to this almost exactly one year later :) I would love to stand on your shoulders. In other words, I would love to reuse all the good Netmiko features of fetching configuration and pushing configuration. I just have to cope with a router that demands a site-specific multi-factor authentication method. Working in bare Paramiko I was able to implement an auth_interactive function that responds to the router appropriately. Just like the Palo Alto router code, my solution subclasses SSHClient. With that extended client in place, Paramkio is able to open a session.

Now I would like to use my subclass in Netmiko. I'll go read that Palo Alto router code a few more times. Do I need to define a class that extends BaseConnection as is done there? Or maybe I should subclass CiscoSSHConnection? In other words, do I essentially need to describe for Netmiko a new type of router, a new string for device_type and insert it somehow into CLASS_MAPPER within ssh_dispatcher.py? I'll go check your doc for an extension guide; i.e., how to provide my code for use by Netmiko.

If you can please find a minute to give tips on how to get started, that sure would accelerate my effort. Thanks in advance.

[chrisinmtown]

The magic of mapping device type to class all seems to be done in ssh_dispatcher.py. The list seems to be static. I do not see a function that lets me dynamically add a device type and associated class. I made a first clumsy attempt at defining one. It's clumsy because it does not update the platforms string. Here is a diff to show you what I'm fumbling with:

diff --git a/netmiko/ssh_dispatcher.py b/netmiko/ssh_dispatcher.py
index 70be1f5b..8193e1ce 100755
--- a/netmiko/ssh_dispatcher.py
+++ b/netmiko/ssh_dispatcher.py
@@ -541,3 +541,8 @@ def FileTransfer(*args: Any, **kwargs: Any) -> "BaseFileTransfer":
     FileTransferClass: Type["BaseFileTransfer"]
     FileTransferClass = FILE_TRANSFER_MAP[device_type]
     return FileTransferClass(*args, **kwargs)
+
+
+def add_device_type(obj: "BaseConnection", device_type: str) -> None:
+    """Add a new device type and associated connection class."""
+    CLASS_MAPPER[device_type] = obj

Since Python allows it, next I'll try doing something crude & ugly like this in my code before I invoke a Netmiko method:

netmiko.ssh_dispatcher.CLASS_MAPPER['my_device_type'] = MyLocalCiscoMFASubclass

Please comment.

[ktbyers]

Yeah, that probably works.

Did you test it and was it successful?

Probably safer/easier to just directly update the CLASS_MAPPER dictionary outside of ssh_dispatcher. This would allow you to upgrade Netmiko without changing the ssh_dispatcher.py file.

If you are not actually going to upgrade Netmiko, i.e. you are just going to have your fork of Netmiko, then probably easiest to just follow the standard Netmiko process for adding a new driver:

https://github.com/ktbyers/netmiko/blob/develop/VENDOR.md

[chrisinmtown]

Thanks for the reply. I got distracted by other things so do not have results to report yet.

To answer your question, I absolutely, positively do NOT want to fork. A fork gives up all the benefit of reuse! I would love to contribute a new driver, but I have to seek employer approval of open-source contributions and that takes time; since it touches security it probably will go nowhere.

I hope you might consider adding some kind of hook function, something that people like me could invoke to add a custom driver without doing crude & ugly things.

[ktbyers]

In the short-term, you should be able just dynamically update CLASS_MAPPER (i.e. it is a mutable dictionary).

Though...it is probably a good idea to provide a way to dynamically add a driver (and to have that way be a part of Netmiko and tested appropriately).

[chrisinmtown]

Thanks for the new title & enhancement label!

[ktbyers]

@chrisinmtown Title change is easy...doing the work, on the other hand :-)

[chrisinmtown]

So far these rather ugly lines seem to be enough to add a custom device type and class to the Netmiko set:

from netmiko.ssh_dispatcher import CLASS_MAPPER, platforms
from cisco_ios_ssh_mfa import CiscoIosSSHWithMFA

DEVICE_TYPE_CISCO_MFA = 'cisco_ios_mfa'

# add the name and class to the netmiko device-type map
CLASS_MAPPER[DEVICE_TYPE_CISCO_MFA] = CiscoIosSSHWithMFA
# add the class name to the netmiko device-types list
platforms.append(DEVICE_TYPE_CISCO_MFA)

I hope to propose a rewrite of the ssh_dispatcher.py module so it doesn't initialize everything just as part of its general startup, instead maybe exposes a function for this, just have not dug into that yet.

[chrisinmtown]

I propose adding a function to ssh_dispatcher.py to allow easy extension of Netmiko with a custom SSH platform driver. As written this requires absolutely no change to the existing API or module structure. Please consider:

def add_platform(
        device_type: str,
        device_class: Type[Any]) -> None:
    """
    Extend Netmiko with a custom platform that supports
    SSH connections.

    This is a stop-gap measure; this module should be
    restructured to allow for easier platform extension.

    :param: device_type: Device type name
    :param device_class: Python class that inherits
        from netmiko.base_connection.BaseConnection
    """
    CLASS_MAPPER[device_type] = device_class
    platforms.append(device_type)

There are some caveats here: 1) The error message that includes all the supported platforms does /not/ include the newly added device type 2) The third line really should have this type hint:

        device_class: Type[BaseConnection]) -> None:

Didn't work in the current version, I see that the module only imports BaseConnection if symbol TYPE_CHECKING is defined/True, and I didn't want to change anything there.

[chrisinmtown]

I have a new, simpler change to propose to ssh_dispatcher.py for using a custom class that inherits from BaseConnection. Here's a diff including a brief Pydoc revision:

diff --git a/netmiko/ssh_dispatcher.py b/netmiko/ssh_dispatcher.py
index 70be1f5b..93537cd9 100755
--- a/netmiko/ssh_dispatcher.py
+++ b/netmiko/ssh_dispatcher.py
@@ -395,7 +395,13 @@ telnet_platforms_str = "\n" + telnet_platforms_str

 def ConnectHandler(*args: Any, **kwargs: Any) -> "BaseConnection":
-    """Factory function selects the proper class and creates object based on device_type."""
+    """
+    Factory function selects the proper class and creates object based on device_type.
+    To use a custom, unregistered class (must inherit from BaseConnection), pass
+    the class with kwarg "connection_class", which will override device_type.
+    """
+    if ConnectionClass := kwargs.pop("connection_class", None):
+        return ConnectionClass(*args, **kwargs)
     device_type = kwargs["device_type"]
     if device_type not in platforms:
         if device_type is None:

Would you possibly consider accepting this minor extension? Do you absolutely need a PR from me?

[chrisinmtown]

Maybe it's obvious to most users, but it took me a while to notice that the factory function ConnectHandler does not perform any required initializations etc. This means that an unregistered class that inherits from BaseConnection can simply be instantiated and used without ever calling the factory function :) So the change I proposed to ConnectHandler is just a convenience, really more documentation than a required step to use a custom class.

[chrisinmtown]

Please consider adding this new subsection to the COMMON_ISSUES.md file, I hope it helps.

diff --git a/COMMON_ISSUES.md b/COMMON_ISSUES.md
index 0d19beb3..2ea40d08 100644
--- a/COMMON_ISSUES.md
+++ b/COMMON_ISSUES.md
@@ -111,3 +111,75 @@ python3.10 -m venv .venv

 6. source .venv/bin/activate   # Activate virtual env 
 7. poetry install       # Install all of the Netmiko dependencies
+
+
+### My device needs custom behavior, for example authentication
+
+If you need to work with a device that requires a change to the
+platform code included in Netmiko, you can use your own Python
+class and still reuse all the great Netmiko features.  You don't
+have to change Netmiko, and you don't have to contribute your code
+to the project. For example, you might need to implement a custom
+authentication scheme, and provide your own implementation of
+`paramiko.SSHClient` for Netmiko to use.
+
+First, write a Python class that inherits from `netmiko.base_connection.BaseConnection`.
+Override any methods as needed to change the behavior.
+Second, create an instance of that class, and use the instance exactly
+as shown in the Netmiko examples. The key difference is that your code
+will have to translate your custom device-type name to your class. You
+cannot call the factory function `ConnectHandler`, but that is not essential,
+it does not perform any required initializations etc.
+
+Here's a complete example of a custom class:
+
+    class MyCustomCiscoXrSSH(CiscoXrSSH):
+        def __init__(
+                self,
+                **kwargs):
+            """
+            Create a new instance.
+
+            :param kwargs: Keyword arguments must include
+                myparam1 and myparam2 for my custom SSHClient.
+                All other arguments are passed to the superclass;
+                see netmiko.base_connection.BaseConnection for
+                valid constructor parameters.
+            """
+            # pluck off the custom SSH client arguments
+            self.myparam1 = kwargs.pop('myparam1')
+            self.myparam2 = kwargs.pop('myparam2')
+            super().__init__(**kwargs)
+
+    def _build_ssh_client(self) -> SSHClient:
+        """
+        Replace the function that prepares a Paramiko SSH connection.
+        """
+        # Create instance of our custom SSHCLient object with
+        # the custom arguments
+        remote_conn_pre: SSHClient = MyCustomSSHClient(
+            self.myparam1, self.myparam2)
+
+        # REMAINING LINES ARE UNCHANGED FROM THE SUPERCLASS
+
+        # Load host_keys for better SSH security
+        if self.system_host_keys:
+            remote_conn_pre.load_system_host_keys()
+        if self.alt_host_keys and path.isfile(self.alt_key_file):
+            remote_conn_pre.load_host_keys(self.alt_key_file)
+        # Default is to automatically add untrusted hosts
+        # (make sure appropriate for your env)
+        remote_conn_pre.set_missing_host_key_policy(self.key_policy)
+        return remote_conn_pre
+
+Use your new custom class like this:
+
+    net_connect = MyCustomCiscoXrSSH(
+        host='host',
+        username='user',
+        password='pass',
+        myparam1='special param one',
+        myparam2='special param two')
+    output = net_connect.send_command(command_string=command)
+    net_connect.disconnect()

[chrisinmtown]

Hi @ktbyers you can ignore the code suggestions, I don't think they add much value. What do you think about the proposed doc addition?

[ktbyers]

I will probably look at some (hopefully) simple solution to dynamically add device_types so that standard ConnectHandler calls work.

There are a bunch of cases where Netmiko code is already integrated in (so overriding it, might not be easy). For example nornir-netmiko.