Error when run ansible playbook fatal: [router]: FAILED! => {"changed": false, "msg": "'int' object has no attribute 'encode'"}

[codecloudmaster]

SUMMARY

Hello, when trying to run playbook from tutorial on two Mikrotik devices with different ROS (6 and 7), the playbook ends with an error. Run it on two diffrent mashines: Ubuntu 22.04.2 (SRV1) and Ubuntu server 20.04.6 (SRV2)

ISSUE TYPE

• Bug Report

COMPONENT NAME

ansible.netcommon.network_cli community.routeros.routeros

ANSIBLE VERSION

SRV1:
ansible [core 2.14.4]
  config file = /home/tolliik/devops/ansible/playbooks/ansible.cfg
  configured module search path = ['/home/tolliik/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/tolliik/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/tolliik/.ansible/collections
  executable location = /home/tolliik/.local/bin/ansible
  python version = 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

SRV2:
ansible [core 2.13.8]
  config file = /home/tolliik/playbooks/ansible.cfg
  configured module search path = ['/home/tolliik/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/tolliik/.local/lib/python3.8/site-packages/ansible
  ansible collection location = /home/tolliik/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/tolliik/.local/bin/ansible
  python version = 3.8.10 (default, Mar 13 2023, 10:26:41) [GCC 9.4.0]
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION

Outpoot of ansible-galaxy collection list ansible.netcommon (I got the same error when the version was only one)

SRV1:
# /home/tolliik/.local/lib/python3.10/site-packages/ansible_collections
Collection        Version
----------------- -------
ansible.netcommon 4.1.0  

# /home/tolliik/.ansible/collections/ansible_collections
Collection        Version
----------------- -------
ansible.netcommon 5.1.1  

SRV2:
# /usr/lib/python3/dist-packages/ansible_collections
Collection        Version
----------------- -------
ansible.netcommon 2.6.1  

# /home/tolliik/.local/lib/python3.8/site-packages/ansible_collections
Collection        Version
----------------- -------
ansible.netcommon 3.1.3  

# /usr/local/lib/python3.8/dist-packages/ansible_collections
Collection        Version
----------------- -------
ansible.netcommon 3.1.3  

# /home/tolliik/.ansible/collections/ansible_collections
Collection        Version
----------------- -------
ansible.netcommon 5.1.1

CONFIGURATION

SRV1:
COLLECTIONS_PATHS(/home/tolliik/devops/ansible/playbooks/ansible.cfg) = ['/home/tolliik/.ansible/collections']
CONFIG_FILE() = /home/tolliik/devops/ansible/playbooks/ansible.cfg
DEFAULT_DEBUG(env: ANSIBLE_DEBUG) = True
DEFAULT_LOG_PATH(env: ANSIBLE_LOG_PATH) = /home/tolliik/ansible.log
PARAMIKO_HOST_KEY_AUTO_ADD(/home/tolliik/devops/ansible/playbooks/ansible.cfg) = True
SRV2:
PARAMIKO_HOST_KEY_AUTO_ADD(/home/tolliik/playbooks/ansible.cfg) = True

OS / ENVIRONMENT

Mikrotik ROS 6.49.8 and Mikrotik ROS 7.9

STEPS TO REPRODUCE

hosts file: [routers] router ansible_host=192.168.1.34

[routers:vars] ansible_connection=ansible.netcommon.network_cli ansible_network_os=community.routeros.routeros ansible_user=***** ansible_ssh_pass=****

---
- name: RouterOS test with network_cli connection
  hosts: routers
  gather_facts: false
  tasks:

  - name: Gather system resources
    community.routeros.command:
      commands:
        - /system resource print
    register: system_resource_print

  - name: Show system resources
    debug:
      var: system_resource_print.stdout_lines

  - name: Gather facts
    community.routeros.facts:

  - name: Show a fact
    debug:
      msg: "First IP address: {{ ansible_net_all_ipv4_addresses[0] }}"

EXPECTED RESULTS

PLAY [RouterOS test with network_cli connection] *****************************************************************

TASK [Gather system resources] ***********************************************************************************
ok: [router]

TASK [Show system resources] *************************************************************************************
ok: [router] => {
    "system_resource_print.stdout_lines": [
        [
            "uptime: 3d10h28m51s",
            "                  version: 6.48.3 (stable)",
            "               build-time: May/25/2021 06:09:45",
            "              free-memory: 31.2MiB",
            "             total-memory: 64.0MiB",
            "                      cpu: MIPS 24Kc V7.4",
            "                cpu-count: 1",
            "            cpu-frequency: 400MHz",
            "                 cpu-load: 1%",
            "           free-hdd-space: 54.2MiB",
            "          total-hdd-space: 128.0MiB",
            "  write-sect-since-reboot: 927",
            "         write-sect-total: 51572981",
            "               bad-blocks: 1%",
            "        architecture-name: mipsbe",
            "               board-name: RB750GL",
            "                 platform: MikroTik"
        ]
    ]
}

TASK [Gather facts] **********************************************************************************************
ok: [router]

TASK [Show a fact] ***********************************************************************************************
ok: [router] => {
    "msg": "First IP address: 192.168.2.1"
}

PLAY RECAP *******************************************************************************************************
router                     : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

ACTUAL RESULTS

For example from SRV2

ansible-playbook [core 2.13.8]
  config file = /home/tolliik/playbooks/ansible.cfg
  configured module search path = ['/home/tolliik/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/tolliik/.local/lib/python3.8/site-packages/ansible
  ansible collection location = /home/tolliik/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/tolliik/.local/bin/ansible-playbook
  python version = 3.8.10 (default, Mar 13 2023, 10:26:41) [GCC 9.4.0]
  jinja version = 3.1.2
  libyaml = True
Using /home/tolliik/playbooks/ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /home/tolliik/playbooks/hosts as it did not pass its verify_file() method
script declined parsing /home/tolliik/playbooks/hosts as it did not pass its verify_file() method
auto declined parsing /home/tolliik/playbooks/hosts as it did not pass its verify_file() method
Parsed /home/tolliik/playbooks/hosts inventory source with ini plugin
Loading collection community.routeros from /home/tolliik/.ansible/collections/ansible_collections/community/routeros
Loading callback plugin default of type stdout, v2.0 from /home/tolliik/.local/lib/python3.8/site-packages/ansible/plugins/callback/default.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: mikrotik.yml ************************************************************************************************************************************************
Positional arguments: mikrotik.yml
verbosity: 4
connection: smart
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/home/tolliik/playbooks/hosts',)
forks: 5
1 plays in mikrotik.yml

PLAY [RouterOS test with network_cli connection] **********************************************************************************************************************
META: ran handlers

TASK [Gather system resources] ****************************************************************************************************************************************
task path: /home/tolliik/playbooks/mikrotik.yml:7
Loading collection ansible.netcommon from /home/tolliik/.ansible/collections/ansible_collections/ansible/netcommon
Loading collection ansible.utils from /home/tolliik/.ansible/collections/ansible_collections/ansible/utils
<192.168.1.34> attempting to start connection
<192.168.1.34> using connection plugin ansible.netcommon.network_cli
Found ansible-connection at path /home/tolliik/.local/bin/ansible-connection
<192.168.1.34> local domain socket does not exist, starting it
<192.168.1.34> control socket path is /home/tolliik/.ansible/pc/e93e03f2f2
<192.168.1.34> Loading collection ansible.netcommon from /home/tolliik/.ansible/collections/ansible_collections/ansible/netcommon
<192.168.1.34> Loading collection ansible.utils from /home/tolliik/.ansible/collections/ansible_collections/ansible/utils
<192.168.1.34> Loading collection community.routeros from /home/tolliik/.ansible/collections/ansible_collections/community/routeros
<192.168.1.34> local domain socket listeners started successfully
<192.168.1.34> loaded cliconf plugin ansible_collections.community.routeros.plugins.cliconf.routeros from path /home/tolliik/.ansible/collections/ansible_collections/community/routeros/plugins/cliconf/routeros.py for network_os community.routeros.routeros
<192.168.1.34> ssh type is set to auto
<192.168.1.34> autodetecting ssh_type
<192.168.1.34> ssh type is now set to libssh
<192.168.1.34> 
<192.168.1.34> local domain socket path is /home/tolliik/.ansible/pc/e93e03f2f2
<192.168.1.34> ESTABLISH LOCAL CONNECTION FOR USER: tolliik
<192.168.1.34> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/tolliik/.ansible/tmp/ansible-local-57510kjlf8v61 `"&& mkdir "` echo /home/tolliik/.ansible/tmp/ansible-local-57510kjlf8v61/ansible-tmp-1685096668.4424844-57515-191673264017952 `" && echo ansible-tmp-1685096668.4424844-57515-191673264017952="` echo /home/tolliik/.ansible/tmp/ansible-local-57510kjlf8v61/ansible-tmp-1685096668.4424844-57515-191673264017952 `" ) && sleep 0'
Using module file /home/tolliik/.ansible/collections/ansible_collections/community/routeros/plugins/modules/command.py
<192.168.1.34> PUT /home/tolliik/.ansible/tmp/ansible-local-57510kjlf8v61/tmp8r6nxezx TO /home/tolliik/.ansible/tmp/ansible-local-57510kjlf8v61/ansible-tmp-1685096668.4424844-57515-191673264017952/AnsiballZ_command.py
<192.168.1.34> EXEC /bin/sh -c 'chmod u+x /home/tolliik/.ansible/tmp/ansible-local-57510kjlf8v61/ansible-tmp-1685096668.4424844-57515-191673264017952/ /home/tolliik/.ansible/tmp/ansible-local-57510kjlf8v61/ansible-tmp-1685096668.4424844-57515-191673264017952/AnsiballZ_command.py && sleep 0'
<192.168.1.34> EXEC /bin/sh -c '/usr/bin/python3 /home/tolliik/.ansible/tmp/ansible-local-57510kjlf8v61/ansible-tmp-1685096668.4424844-57515-191673264017952/AnsiballZ_command.py && sleep 0'
<192.168.1.34> EXEC /bin/sh -c 'rm -f -r /home/tolliik/.ansible/tmp/ansible-local-57510kjlf8v61/ansible-tmp-1685096668.4424844-57515-191673264017952/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
  File "/tmp/ansible_community.routeros.command_payload_aheplf2u/ansible_community.routeros.command_payload.zip/ansible_collections/community/routeros/plugins/module_utils/routeros.py", line 51, in get_capabilities
    capabilities = Connection(module._socket_path).get_capabilities()
  File "/tmp/ansible_community.routeros.command_payload_aheplf2u/ansible_community.routeros.command_payload.zip/ansible/module_utils/connection.py", line 200, in __rpc__
    raise ConnectionError(to_text(msg, errors='surrogate_then_replace'), code=code)
fatal: [router]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "commands": [
                "/system resource print"
            ],
            "interval": 1,
            "match": "all",
            "retries": 10,
            "wait_for": null
        }
    },
    "msg": "'int' object has no attribute 'encode'"
}

PLAY RECAP ************************************************************************************************************************************************************
router                     : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0 

[felixfontein]

It looks like the real source of the error got lost in the network_cli wrapping code. I don't see any encode() calls in community.routeros, which makes this more confusing.

[BSMaximN]

@felixfontein Will they fix the problem?

OS: Ubuntu 22.04 Version plugin: routeros 2.8.0 Using module ssh: paramiko-3.2.0

The full traceback is:
  File "/tmp/ansible_community.routeros.command_payload_peiaka_2/ansible_community.routeros.command_payload.zip/ansible_collections/community/routeros/plugins/module_utils/routeros.py", line 51, in get_capabilities
    capabilities = Connection(module._socket_path).get_capabilities()
  File "/tmp/ansible_community.routeros.command_payload_peiaka_2/ansible_community.routeros.command_payload.zip/ansible/module_utils/connection.py", line 200, in __rpc__
    raise ConnectionError(to_text(msg, errors='surrogate_then_replace'), code=code)
fatal: [router]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "commands": [
                "/system resource print"
            ],
            "interval": 1,
            "match": "all",
            "retries": 10,
            "wait_for": null
        }
    },
    "msg": "Expected unicode or bytes, got <class 'int'>"
}

[felixfontein]

I guess someone needs to debug this first to figure out what exactly the problem is. I cannot reproduce the problem locally, so I cannot do that. Unfortunately debugging this is harder than regular debugging, since the error likely happens somewhere in ansible.netcommon or ansible-core, and since network_cli uses some complicated mechanism to actually run things. If I could reproduce this I would try to add a lot more q.q() statements (with this q) in various places to try to figure out where the exception actually comes from.

[codecloudmaster]

HI. Sorry for the long response. On Ubuntu 20.04 I did not have this error. I will try to do it on a clean Ubuntu 22.04 and then will write the result here.

[codecloudmaster]

Even more. I try it on another router and Ubuntu 22.04 and still do not get an error, so I think maybe the problem is on my Mikrotik conf side. If I found it I will write the result here.

[BSMaximN]

@felixfontein I reproduced and made video. https://disk.yandex.ru/i/kRNa4d0FzoxzBQ

[felixfontein]

@BSMaximN thanks for the video, but to be able to debug this I need direct access to an environment where this error happens. A video unfortunately does not help.

@itcrowd007 it could be the router config, the ansible.netcommon version, the ansible-core version, the community.routeros version, the Python version, or a combination of these. I hope you can figure out some combination that might be reproducable somewhere else :)

[BSMaximN]

@felixfontein @itcrowd007 The problem is in the module paramiko/util.py", line 326, because the function b does not know how to process variable integer

paramiko [192.168.9.41] /tmp/community.routeros/venv/lib/python3.10/site-packages/paramiko/util.py", line 326, in b
paramiko [192.168.9.41] ERROR     raise TypeError(f"Expected unicode or bytes, got {type(s)}")
paramiko [192.168.9.41] ERROR TypeError: Expected unicode or bytes, got <class 'int'>

I entered the password in invertory.cfg ansible_ssh_pass=1234567890 Solution(make the password a string): ansible_ssh_pass='1234567890'

[felixfontein]

Ah. I was able to reproduce that, with libssh though; that results in 'int' object has no attribute 'encode' as reported by @itcrowd007. That can be fixed in ansible.netcommon (or by forcing the password to be a string with quotes, as you did). I'm currently trying to get paramiko to work to see where it needs to be fixed...

[felixfontein]

(Either ansible-core itself or ansible.netcommon as well?)

[felixfontein]

https://github.com/ansible-collections/ansible.netcommon/pull/549 fixes the problem for me (with libssh), it might also fix the problem with paramiko since it makes sure that network_cli obtains the password as a string (and not as the type which ansible-core read from the inventory).

[felixfontein]

I also created a similar PR for ansible-core, this might fix the paramiko issue: https://github.com/ansible/ansible/pull/81029

[codecloudmaster]

Hi. I can confirm too: the simple solution from @felixfontein works.

I entered the password in hosts.ini ansible_pasword='1234567890'

Thank you. I should have tried it before

[Ahmadshefayee]

actually in the previous versions using pythonparamiko when using ansible, that was fine adding passwords without Quots but in the new version that use pylibssh you will got this error: FAILED! => {"changed": false, "msg": "'int' object has no attribute 'encode'"} and you should put your passwords between quotes: in the file hosts

[devices] 192.168.33.40 [devices:vars] ansible_become=yes ansible_become_method=enable ansible_network_os=ios ansible_user=your cisco device username ansible_ssh_pass='12345678' ansible_become_pass='12345678'