search

ktbyers / netmiko

Multi-vendor library to simplify Paramiko SSH connections to network devices
MIT License
3.5k stars 1.27k forks source link

unable to send commands to sophos ssh through netmiko due to readtimeout error. #3362

Open capmerah opened 6 months ago

capmerah commented 6 months ago

Description of Issue/Question

I received the read timeout error when I run the netmiko to connect to sophos firewall ssh. The initial output for sophos after login is like this:

Sophos Firmware Version: SFOS 20.0.0 GA-Build222
Model: SG430
Hostname: companyname.com
HA node name: Node1 
Current status: Primary (Active) from 04:10:13 PM, Dec 03, 2023 

Main Menu

    1.  Network  Configuration
    2.  System   Configuration
    3.  Route    Configuration
    4.  Device Console
    5.  Device Management
    6.  VPN Management
    7.  Shutdown/Reboot Device
    0.  Exit

    Select Menu Number [0-7]:

I tried ":" as expected string but still the same issue. I tried "Select Menu Number [0-7]:" as expected string but still the same issue.

Note: Please check https://guides.github.com/features/mastering-markdown/ to see how to properly format your request.

Setup

Netmiko version

pip3 freeze | FINDSTR netmiko
netmiko==4.3.0

Netmiko device_type (if relevant to the issue)

(Paste device_type between quotes below)

sophos_sfos_ssh

Steps to Reproduce the Issue

run and debug python code from visual studio code

Error Traceback

(Paste the complete traceback of the exception between quotes below)

Exception has occurred: ReadTimeout

Pattern not detected: '(\\#|>)' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

  File "C:\Users\aa\test\sophos-netmiko-test.py", line 26, in <module>
    with ConnectHandler(**device) as net_connect:
netmiko.exceptions.ReadTimeout: 

Pattern not detected: '(\\#|>)' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

Relevant Python code

(Please try to essentialize your Python code to the minimum code needed to reproduce the issue) (Paste the code between the quotes below)

import getpass
from netmiko import ConnectHandler
from getpass import getpass

#config #1, capture ssl vpn logs into terminal

password = getpass()

ipaddrs = ["10.2.2.254"]

devices = [
    {
        "device_type": "sophos_sfos_ssh",
        "host": ip,
        "username": "admin",
        "password": password,
    }
    for ip in ipaddrs
]

for device in devices:
    print(f'Connecting to the device: {device["host"]}')

    with ConnectHandler(**device) as net_connect:  
        Device_Management = net_connect.send_command("5", expect_string=":")  
        Advanced_Shell = net_connect.send_command("3", expect_string=r":")  
        sslvpn_logs = net_connect.send_command("tail -f /log/sslvpn.log", expect_string=r"#")  

    print(Device_Management)
    print(Advanced_Shell)
    print(sslvpn_logs)
ktbyers commented 6 months ago

Netmiko automatically sends "4" to the above menu:

https://github.com/ktbyers/netmiko/blob/develop/netmiko/sophos/sophos_sfos_ssh.py#L30

So that menu is no longer present (when you try to send a "5" or a "3".

capmerah commented 6 months ago

Netmiko automatically sends "4" to the above menu:

https://github.com/ktbyers/netmiko/blob/develop/netmiko/sophos/sophos_sfos_ssh.py#L30

So that menu is no longer present (when you try to send a "5" or a "3".

Is there any way to get to the advanced shell, or is it not allowed?

ktbyers commented 6 months ago

@capmerah Does the CLI allow you to re-launch the menu or to switch contexts (i.e. can you go from the "console" selection to the "Device Management" selection in the Sophos CLI)?

capmerah commented 6 months ago

@ktbyers The CLI only allows exit to the main menu from the "Device Console" mode by typing "exit". It cannot go from the "console" selection to the "Device Management" selection.

capmerah commented 6 months ago

Hi @ktbyers, I tried to test with current code (self.write_channel("4" + self.RETURN)) that you've highlighted in your first reply,

for device in devices:
    print(f'Connecting to the device: {device["host"]}')
    with ConnectHandler(**device) as net_connect:  
        Device_Management = net_connect.send_command("show vpn connection status", expect_string=r">")
    print(Device_Management)

since it sends "4" automatically, I tried to input "show vpn connection status" which should output the logs in the cli, with expect_string=r">". but it still gives the same error.

ktbyers commented 6 months ago

@capmerah Which problem do you want to work on the original problem or the second problem?

If the second problem, please include your full exception stack trace for that case.

capmerah commented 6 months ago

Hi @ktbyers , sorry about the confusion. We can try to solve the first problem first. If we can't, we can then move on to the second problem and I'll post full exception stack.

ktbyers commented 6 months ago

@capmerah Sounds good.

The CLI only allows exit to the main menu from the "Device Console" mode by typing "exit". It cannot go from the "console" selection to the "Device Management" selection.

So can you just use Netmiko to send exit and then send the numbers you want?

When you select Device Management -> Advanced Shell, what does the SSH interaction look like?

capmerah commented 6 months ago

Hi @ktbyers , thanks for your understanding. I tried with the following code to add 'exit' into the line to try to exit from the main menu followed by numbers: 

from netmiko import ConnectHandler
from getpass import getpass

password = getpass()

ipaddrs = ["10.2.2.254"]

devices = [
    {
        "device_type": "sophos_sfos_ssh",
        "host": ip,
        "username": "admin",
        "password": password
    }
    for ip in ipaddrs
]

for device in devices:
    print(f'Connecting to the device: {device["host"]}')

    with ConnectHandler(**device) as net_connect:  
        Main_Menu = net_connect.send_command("exit", expect_string=r"console> ")
        Device_Management = net_connect.send_command("5", expect_string=r"    Select Menu Number \[0-7\]: ")
        Advanced_Shell = net_connect.send_command("3", expect_string=r"    Select Menu Number \[0-4\]: ")

        #sslvpn_logs = net_connect.send_command("tail -f /log/sslvpn.log", expect_string=r"#") 

    print(Main_Menu)
    print(Device_Management)
    print(Advanced_Shell)

But I received the same exceptions about read timeout error due to wrong patterns, even though I changed the expect_string to ">" for main menu and ":" for the rest.

Exception has occurred: ReadTimeout

Pattern not detected: '(\\#|>)' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

  File "C:\Users\aa\companyname\aaa\sophos-netmiko-test.py", line 23, in <module>
    with ConnectHandler(**device) as net_connect:

For outputs, here is the output after initial SSH login:

Sophos Firmware Version: SFOS 20.0.0 GA-Build222
Model: SG430
Hostname: Companyname.com
HA node name: Node1 
Current status: Primary (Active) from 04:10:13 PM, Dec 03, 2023 

Main Menu 

    1.  Network  Configuration
    2.  System   Configuration
    3.  Route    Configuration 
    4.  Device Console 
    5.  Device Management
    6.  VPN Management
    7.  Shutdown/Reboot Device
    0.  Exit 

    Select Menu Number [0-7]:

After selecting '5. Device Management', it will immediately prompt to the following menu

Sophos Firmware Version: SFOS 20.0.0 GA-Build222
Model: SG430
Hostname: Companyname.com
HA node name: Node1 
Current status: Primary (Active) from 04:10:13 PM, Dec 03, 2023 

Device Management 

    1.  Reset to Factory Defaults
    2.  Show Firmware(s)
    3.  Advanced Shell
    4.  Flush Device Reports
    0.  Exit

    Select Menu Number [0-4]:

Then, selecting '3. Advanced Shell' will immediately go to the following output:

Sophos Firewall
===============
(C) Copyright 2000-2023 Sophos Limited and others. All rights reserved.
Sophos is a registered trademark of Sophos Limited and Sophos Group.
All other product and company names mentioned are trademarks or registered
trademarks of their respective owners.

For Sophos End User Terms of Use - https://www.sophos.com/en-us/legal/sophos-end-user-terms-of-use.aspx

NOTE: If not explicitly approved by Sophos support, any modifications
      done through this option will void your support.

SG430_WP02_SFOS 20.0.0 GA-Build222 HA-Primary#

This is for SFOS 20.0.0 latest version from Sophos.

ktbyers commented 6 months ago

@capmerah

I would have expected it to be more like this. In other words, you type exit and what it sees next is the "Main Menu". Note, I simplified the regex pattern (i.e. brackets have special meaning in regex so it is easier to not need to escape them).

And then after you hit 3, what you are searching for next is the # in the device prompt.

        output = net_connect.send_command("exit", expect_string=r"Select Menu")
        output += net_connect.send_command("5", expect_string=r"Select Menu")
        output += net_connect.send_command("3", expect_string=r"#")
capmerah commented 6 months ago

Hi @ktbyers , here is the code I modified in accordance to your recommendations:

import getpass
from netmiko import ConnectHandler
from getpass import getpass

password = getpass()

ipaddrs = ["10.2.2.254"]

devices = [
    {
        "device_type": "sophos_sfos_ssh",
        "host": ip,
        "username": "admin",
        "password": password
    }
    for ip in ipaddrs
]

for device in devices:
    print(f'Connecting to the device: {device["host"]}')

    with ConnectHandler(**device) as net_connect:  
        output = net_connect.send_command("exit", expect_string=r"Select Menu")
        output += net_connect.send_command("5", expect_string=r"Select Menu")
        output += net_connect.send_command("3", expect_string=r"#")

    print(output)

The results gives the same exception throughout:

Exception has occurred: ReadTimeout

Pattern not detected: '(\\#|>)' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.
ktbyers commented 6 months ago

@capmerah You need to post the full exception stack trace so I can see where the error is happening.

You also might need to include the Netmiko session_log so we can see more details on what is happening.

Thanks, Kirk

capmerah commented 6 months ago

Hi @ktbyers ,

Full exception is here:

Exception has occurred: ReadTimeout

Pattern not detected: '(\\#|>)' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

  File "C:\Users\aa\companyname\aaa\sophos-netmiko-test.py", line 24, in <module>
    with ConnectHandler(**device) as net_connect:
netmiko.exceptions.ReadTimeout: 

Pattern not detected: '(\\#|>)' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

The following comes from a session logs:



Sophos Firmware Version: SFOS 20.0.0 GA-Build222 
Model: SG430 
Hostname: Companyname.com 
HA node name: Node1 
Current status: Primary (Active) from 04:10:13 PM, Dec 03, 2023 

Main Menu 

    1.  Network  Configuration
    2.  System   Configuration
    3.  Route    Configuration 
    4.  Device Console 
    5.  Device Management
    6.  VPN Management
    7.  Shutdown/Reboot Device
    0.  Exit 

    Select Menu Number [0-7]:      Invalid Menu Selection4

Sophos Firmware Version: SFOS 20.0.0 GA-Build222 
Model: SG430 
Hostname: Companyname.com 
HA node name: Node1 
Current status: Primary (Active) from 04:10:13 PM, Dec 03, 2023 

Main Menu 

    1.  Network  Configuration
    2.  System   Configuration
    3.  Route    Configuration 
    4.  Device Console 
    5.  Device Management
    6.  VPN Management
    7.  Shutdown/Reboot Device
    0.  Exit 

    Select Menu Number [0-7]: 
     Invalid Menu Selectionexit

Thanks and regards, Capmerah

ktbyers commented 6 months ago

It looks like you are rejecting option 4?

    Select Menu Number [0-7]:      Invalid Menu Selection4
�[H�[J

Does your device/configuration not allow you to select 4 (Device Console)?

capmerah commented 6 months ago

Hi @ktbyers , This seems like the Invalid Menu Selection comes from keyboard input that is out of scope of [0-7], meaning, any alphanumeric other than 0 to 7. the unicode characters "�[H�[J" comes from some simulation of gibberish inputs from keyboard which I don't know which keyboard input.

The device/configuration does not reject option 4.

ktbyers commented 6 months ago

This:

"�[H�[J"

is ANSI escape codes which are output from the device (and which netmiko will strip off).

So you tested the same device (shown here) and sent the number 4 and it worked?

I just checked the Netmiko code and it just sends a 4 and an ?

capmerah commented 6 months ago

Hi @ktbyers ,

I manually connect to Sophos through SSH and send '4' and it works

I tried testing netmiko to see if its send 4 and ? with only net_connect:

import getpass
from netmiko import ConnectHandler
from getpass import getpass

password = getpass()

ipaddrs = ["10.2.2.254"]

devices = [
    {
        "device_type": "sophos_sfos_ssh",
        "host": ip,
        "username": "admin",
        "password": password,
        "session_log": 'netmiko_session.log'
    }
    for ip in ipaddrs
]

for device in devices:
    print(f'Connecting to the device: {device["host"]}')

    with ConnectHandler(**device) as net_connect:  
        output = net_connect

    print(output)

It still show the same exception and same session log outputs. It seems that something must've triggered an option that is out of scope before entering '4' and 'exit'.

capmerah commented 6 months ago

@ktbyers The "Invalid Menu Selection" needed to be entered again in order to revert to the main menu and allowing us to select [0-7]. The "Invalid Menu Selection4" and "Invalid Menu Selectionexit" will take no effect and will automatically revert to the main menu.

ktbyers commented 6 months ago

I didn't understand your last comment?

Netmiko fails on the connection (from your stack trace) and session log as it tries to send 4 and expects to read the device's prompt after this (and it never does)...do to the Invalid Menu Selection4.

So the question I still have is why does sending the 4 fail.

capmerah commented 6 months ago

Hi @ktbyers , By checking the session logs, I tested manually to simulate the error. Invalid Menu Selection will prompt when the you press the button that is outside the digits between [0-7]. for example: in the main menu, pressing 8 or 9 or any alphabetical's and then press enter. Invalid Menu Selection will appear.

Invalid Menu Selection will not disappear until you press "enter". So if you see Invalid Menu Selection and then press 4 and then press "enter", it will ignore that command. It will become Invalid Menu Selection4 . That is why sending the 4 fail.

In normal operations when you see Invalid Menu Selection, you have to press "enter" so that it returns to the main menu and then press 4 to go to the device console, not the other way around.

ktbyers commented 6 months ago

Okay, I will see if I can put in a fix...

capmerah commented 6 months ago

Ok, thanks.

ktbyers commented 5 months ago

@capmerah Can you test this fix?

https://github.com/ktbyers/netmiko/pull/3367

ktbyers commented 5 months ago

@capmerah Note, there is an environment variable that you can set:

SOPHOS_MENU_DEFAULT = os.getenv("NETMIKO_SOPHOS_MENU", "4")

In your case, you could probably set this NETMIKO_SOPHOS_MENU environment variable to 5\r3 (note, the environment variable must result in the device returning a prompt (i.e. it will fail if you just hit 5 and you are still at the Main Menu).

capmerah commented 5 months ago

Hi @ktbyers , thanks for the help. The changes I have made to sophos folder according to #3367 with modifications to NETMIKO_SOPHOS_MENU into 5\r3 and before changing back to NETMIKO_SOPHOS_MENU , I have tested both env variables. I also command set NETMIKO_SOPHOS_MENU=5\r3 into cmd before testing:

"""SophosXG (SFOS) Firewall support"""
from typing import Any
import time
import os

from netmiko.no_enable import NoEnable
from netmiko.no_config import NoConfig
from netmiko.cisco_base_connection import CiscoSSHConnection

SOPHOS_MENU_DEFAULT = os.getenv("5\r3", "4")

class SophosSfosSSH(NoEnable, NoConfig, CiscoSSHConnection):
    def session_preparation(self) -> None:
        """Prepare the session after the connection has been established."""
        self._test_channel_read(pattern=r"Main Menu")
        """
        Sophos Firmware Version SFOS 18.0.0 GA-Build339

        Main Menu

            1.  Network  Configuration
            2.  System   Configuration
            3.  Route    Configuration
            4.  Device Console
            5.  Device Management
            6.  VPN Management
            7.  Shutdown/Reboot Device
            0.  Exit

            Select Menu Number [0-7]:
        """
        self.write_channel(SOPHOS_MENU_DEFAULT + self.RETURN)
        self._test_channel_read(pattern=r"[#>]")
        self.set_base_prompt()
        # Clear the read buffer
        time.sleep(0.3 * self.global_delay_factor)
        self.clear_buffer()

    def save_config(self, *args: Any, **kwargs: Any) -> str:
        """Not Implemented"""
        raise NotImplementedError

Running with the main code and test multiple commands and expect_strings

import getpass
from netmiko import ConnectHandler
from getpass import getpass

password = getpass()

ipaddrs = ["10.2.2.254"]

devices = [
    {
        "device_type": "sophos_sfos_ssh",
        "host": ip,
        "username": "admin",
        "password": password,
        "session_log": 'netmiko_session.log'
    }
    for ip in ipaddrs
]

for device in devices:
    print(f'Connecting to the device: {device["host"]}')

    with ConnectHandler(**device) as net_connect:  
        #output = net_connect.send_command("tail -f /log/sslvpn.log")
        #output = net_connect.send_command("5\r3")
        #output = net_connect.send_command("l", expect_string=r"#")
        output = net_connect.send_command("")

    print(output)

All of the test yields same exception results:

Exception has occurred: ReadTimeout

Pattern not detected: '[#>]' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

  File "C:\Users\Cap\OneDrive\aa\sophos-netmiko-test.py", line 24, in <module>
    with ConnectHandler(**device) as net_connect:
       ^^^^^^^^^^^^^^^^^^^^^^^^
netmiko.exceptions.ReadTimeout: 

Pattern not detected: '[#>]' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

And all test yields the same session log output:



Sophos Firmware Version: SFOS 20.0.0 GA-Build222 
Model: SG430 
Hostname: Companyname.com 
HA node name: Node1 
Current status: Primary (Active) from 04:10:13 PM, Dec 03, 2023 

Main Menu 

    1.  Network  Configuration
    2.  System   Configuration
    3.  Route    Configuration 
    4.  Device Console 
    5.  Device Management
    6.  VPN Management
    7.  Shutdown/Reboot Device
    0.  Exit 

    Select Menu Number [0-7]:      Invalid Menu Selection4

Sophos Firmware Version: SFOS 20.0.0 GA-Build222 
Model: SG430 
Hostname: Companyname.com 
HA node name: Node1 
Current status: Primary (Active) from 04:10:13 PM, Dec 03, 2023 

Main Menu 

    1.  Network  Configuration
    2.  System   Configuration
    3.  Route    Configuration 
    4.  Device Console 
    5.  Device Management
    6.  VPN Management
    7.  Shutdown/Reboot Device
    0.  Exit 

    Select Menu Number [0-7]: exit

One thing I noticed that has changed is Invalid Menu Selectionexit which becomes Select Menu Number [0-7]: exit

ktbyers commented 5 months ago

Can you test this?



        # I changed the pattern here
        self._test_channel_read(pattern=r"Select Menu Number")

        """
        Sophos Firmware Version SFOS 18.0.0 GA-Build339

        Main Menu

            1.  Network  Configuration
            2.  System   Configuration
            3.  Route    Configuration
            4.  Device Console
            5.  Device Management
            6.  VPN Management
            7.  Shutdown/Reboot Device
            0.  Exit

            Select Menu Number [0-7]:
        """

        # Added time.sleep

        import time
        time.sleep(2)
        self.write_channel(SOPHOS_MENU_DEFAULT + self.RETURN)
        self._test_channel_read(pattern=r"[#>]")
capmerah commented 5 months ago

Hi @ktbyers , Followed the recent change and tested:

"""SophosXG (SFOS) Firewall support"""
from typing import Any
import time
import os

from netmiko.no_enable import NoEnable
from netmiko.no_config import NoConfig
from netmiko.cisco_base_connection import CiscoSSHConnection

SOPHOS_MENU_DEFAULT = os.getenv("5\r3", "4") #"NETMIKO_SOPHOS_MENU"

class SophosSfosSSH(NoEnable, NoConfig, CiscoSSHConnection):
    def session_preparation(self) -> None:
        """Prepare the session after the connection has been established."""
        # I changed the pattern here
        self._test_channel_read(pattern=r"Select Menu Number")

        """
        Sophos Firmware Version SFOS 18.0.0 GA-Build339

        Main Menu

            1.  Network  Configuration
            2.  System   Configuration
            3.  Route    Configuration
            4.  Device Console
            5.  Device Management
            6.  VPN Management
            7.  Shutdown/Reboot Device
            0.  Exit

            Select Menu Number [0-7]:
        """

        # Added time.sleep

        import time
        time.sleep(2)
        self.write_channel(SOPHOS_MENU_DEFAULT + self.RETURN)
        self._test_channel_read(pattern=r"[#>]")
        self.set_base_prompt()
        # Clear the read buffer
        time.sleep(0.3 * self.global_delay_factor)
        self.clear_buffer()

    def save_config(self, *args: Any, **kwargs: Any) -> str:
        """Not Implemented"""
        raise NotImplementedError

ran with my main code:

import getpass
from netmiko import ConnectHandler
from getpass import getpass

password = getpass()

ipaddrs = ["10.2.2.254"]

devices = [
    {
        "device_type": "sophos_sfos_ssh",
        "host": ip,
        "username": "admin",
        "password": password,
        "session_log": 'netmiko_session.log'
    }
    for ip in ipaddrs
]

for device in devices:
    print(f'Connecting to the device: {device["host"]}')

    with ConnectHandler(**device) as net_connect:  
        #output = net_connect.send_command("tail -f /log/sslvpn.log")
        #output = net_connect.send_command("5\r3")
        #output = net_connect.send_command("l", expect_string=r"#")
        #output = net_connect.send_command("ls")
        #output = net_connect.send_command("ls", expect_string=r"#")
        output = net_connect.send_command("")

    print(output)

Same exceptions as before:

Exception has occurred: ReadTimeout

Pattern not detected: '[#>]' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

  File "C:\Users\Capme\OneDrive - company\aa\sophos-netmiko-test.py", line 24, in <module>
    with ConnectHandler(**device) as net_connect:
       ^^^^^^^^^^^^^^^^^^^^^^^^
netmiko.exceptions.ReadTimeout: 

Pattern not detected: '[#>]' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

No changes from session logs:



Sophos Firmware Version: SFOS 20.0.0 GA-Build222 
Model: SG430 
Hostname: Companyname.com 
HA node name: Node1 
Current status: Primary (Active) from 04:10:13 PM, Dec 03, 2023 

Main Menu 

    1.  Network  Configuration
    2.  System   Configuration
    3.  Route    Configuration 
    4.  Device Console 
    5.  Device Management
    6.  VPN Management
    7.  Shutdown/Reboot Device
    0.  Exit 

    Select Menu Number [0-7]:      Invalid Menu Selection4

Sophos Firmware Version: SFOS 20.0.0 GA-Build222 
Model: SG430 
Hostname: Companyname.com 
HA node name: Node1 
Current status: Primary (Active) from 04:10:13 PM, Dec 03, 2023 

Main Menu 

    1.  Network  Configuration
    2.  System   Configuration
    3.  Route    Configuration 
    4.  Device Console 
    5.  Device Management
    6.  VPN Management
    7.  Shutdown/Reboot Device
    0.  Exit 

    Select Menu Number [0-7]: exit
capmerah commented 5 months ago

Hi, Anything for me to test?

ktbyers commented 2 months ago

@capmerah Did you ever get this working?

In your above test-code this is wrong:

SOPHOS_MENU_DEFAULT = os.getenv("5\r3", "4") #"NETMIKO_SOPHOS_MENU"

It should be:

SOPHOS_MENU_DEFAULT = "5\r3"
capmerah commented 1 month ago

Hi @ktbyers ,

Similar results as of SOPHOS_MENU_DEFAULT = "5\r3" , with and without expected strings.

Exception has occurred: ReadTimeout

Pattern not detected: '[#>]' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

  File "C:\Users\username\Companyname\test\Backup config\Sophos Firewall\Netmiko\sophos-netmiko-test-without-getpass.py", line 24, in <module>
    with ConnectHandler(**device) as net_connect:
       ^^^^^^^^^^^^^^^^^^^^^^^^
netmiko.exceptions.ReadTimeout: 

Pattern not detected: '[#>]' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.
ktbyers commented 1 month ago

@capmerah Can you post your code and your session_log?

Thanks, Kirk

capmerah commented 1 month ago

Hi @ktbyers , Here is the code below executing the script:

#import getpass
from netmiko import ConnectHandler
#from getpass import getpass
import os
log_directory = r"C:\User\username\OneDrive - company\username\Backup config\Sophos Firewall\Netmiko"
os.makedirs(log_directory, exist_ok=True)
ipaddrs = [f"10.2.2.254"]

#password = getpass()

ipaddrs = ["10.2.2.254"]

devices = [
    {
        "device_type": "sophos_sfos_ssh",
        "host": ip,
        "username": "admin",
        "password": "password",
        "session_log": os.path.join(log_directory, f"netmiko_session_{ip}.log"),
    }
    for ip in ipaddrs
]

for device in devices:
    print(f'Connecting to the device: {device["host"]}')

    with ConnectHandler(**device) as net_connect:  
        #output = net_connect.send_command("\r", expect_string=r"    Select Menu Number [0-7]:      Invalid Menu Selection4")
        #output = net_connect.send_command("5\r3", expect_string=r"#>")
        #output += net_connect.send_command("tail -f /log/sslvpn.log", expect_string=r"#>")
        #output += net_connect.send_command("ls", expect_string=r"#>")
        output = net_connect.send_command("", expect_string=r"#>")

    print(output)

Below is the session_log after the errors from executing above:

 Sophos Firmware Version: SFOS 20.0.0 GA-Build222 Model: SG430 Hostname: companyname.com HA node name: Node1 Current status: Primary (Active) from 06:28:33 PM, May 03, 2024

Main Menu 

1.  Network  Configuration
2.  System   Configuration
3.  Route    Configuration 
4.  Device Console 
5.  Device Management
6.  VPN Management
7.  Shutdown/Reboot Device
0.  Exit 

Select Menu Number [0-7]:      Invalid Menu Selection5

3  Sophos Firmware Version: SFOS 20.0.0 GA-Build222 Model: SG430 Hostname: companyname.com HA node name: Node1 Current status: Primary (Active) from 06:28:33 PM, May 03, 2024

Main Menu 

1.  Network  Configuration
2.  System   Configuration
3.  Route    Configuration 
4.  Device Console 
5.  Device Management
6.  VPN Management
7.  Shutdown/Reboot Device
0.  Exit 

Select Menu Number [0-7]: 

Sophos Firmware Version: SFOS 20.0.0 GA-Build222 Model: SG430 Hostname: companyname.com HA node name: Node1 Current status: Primary (Active) from 06:28:33 PM, May 03, 2024

Router Management 

1.  Configure Unicast Routing
2.  Configure Multicast Routing
0.  Exit

Select Menu Number [0-2]: exit

Thanks and regards, @capmerah

ktbyers commented 1 month ago

Any idea, why it responds with 5 as being an invalid menu selection?

capmerah commented 1 month ago

Hi @ktbyers , It seems that 5 comes from SOPHOS_MENU_DEFAULT = "5\r3" . When I change from 5 to 4 in SOPHOS_MENU_DEFAULT = "4\r3" , then the output from the session_log will return 4 after invalid menu selection.

I think the self._test_channel_read(pattern=r"Select Menu Number") triggered an invalid menu selection before going to SOPHOS_MENU_DEFAULT = "5\r3".

ktbyers commented 1 month ago

@capmerah

Let's step back for a second.

You should be using the Netmiko develop branch code (unmodified).

You should also be setting the following environment variable (here is how I would do this on Linux, macOS should be similar, you would need to look it up, if you are on Windows):

$ export NETMIKO_SOPHOS_MENU="5\r3\r"

Note, it is not clear to me whether you have to hit the (i.e. the \r) after the 5 (or just hitting the 5 and then the 3 are sufficient). In other words, you would need to test this on the Sophos device.

Because of the above it is possible, that you might set this environment variable to:

# No <enters> just send a '5' and then a '3'
$ export NETMIKO_SOPHOS_MENU="53"

Your test code should just be:


from netmiko import ConnectHandler

# Just manually verify the directory exists and create it (if necessary)
ip = "10.2.2.254"
s_log = rf"C:\User\username\OneDrive - company\username\Backup config\Sophos Firewall\Netmiko\netmiko_session_{ip}.log"

device = {
        "device_type": "sophos_sfos_ssh",
        "host": "10.2.2.254",
        "username": "admin",
        "password": "password",
        "session_log": s_log,
 }

print(f'Connecting to the device: {device["host"]}')

with ConnectHandler(**device) as net_connect:  
        # You should send a real Sophos command here
        output = net_connect.send_command("\n", expect_string=r"#>")
        print(output)

If you do the above, what do you see in the session_log?

capmerah commented 1 month ago

Hi @ktbyers ,

Apologies for late reply due to other projects.

I have downloaded the develop branch code and clone into the netmiko file location for testing.

I have using setx for path in windows setx NETMIKO_SOPHOS_MENU "5\r3\r" and subsequently setx NETMIKO_SOPHOS_MENU "53"

I saved a session_log for each path.

The results are the same for both paths according to the session_log:

 Sophos Firmware Version: SFOS 20.0.0 GA-Build222 Model: SG430 Hostname: cwtgl-fw.globelink-group.com HA node name: Node1 Current status: Primary (Active) from 06:28:33 PM, May 03, 2024

Main Menu 

1.  Network  Configuration
2.  System   Configuration
3.  Route    Configuration 
4.  Device Console 
5.  Device Management
6.  VPN Management
7.  Shutdown/Reboot Device
0.  Exit 

Select Menu Number [0-7]:      Invalid Menu Selection5

3  Sophos Firmware Version: SFOS 20.0.0 GA-Build222 Model: SG430 Hostname: cwtgl-fw.globelink-group.com HA node name: Node1 Current status: Primary (Active) from 06:28:33 PM, May 03, 2024

Main Menu 

1.  Network  Configuration
2.  System   Configuration
3.  Route    Configuration 
4.  Device Console 
5.  Device Management
6.  VPN Management
7.  Shutdown/Reboot Device
0.  Exit 

Select Menu Number [0-7]: 

Sophos Firmware Version: SFOS 20.0.0 GA-Build222 Model: SG430 Hostname: cwtgl-fw.globelink-group.com HA node name: Node1 Current status: Primary (Active) from 06:28:33 PM, May 03, 2024

Router Management 

1.  Configure Unicast Routing
2.  Configure Multicast Routing
0.  Exit

Select Menu Number [0-2]: exit