search

Juniper / ansible-junos-stdlib

Junos modules for Ansible
Apache License 2.0
302 stars 156 forks source link

juniper.device.config always performs a commit confirmed operation #660

Closed slefol closed 4 months ago

slefol commented 4 months ago

Issue Type

Module Name

juniper.device.config

juniper.device collection and Python libraries version

$ ansible --version
ansible [core 2.16.6]
  config file = /home/ansible/ansible_network/ansible.cfg
  configured module search path = ['/home/xxx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/xxx/ansible-venv/lib/python3.11/site-packages/ansible
  ansible collection location = /home/xxx/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/xxx/ansible-venv/bin/ansible
  python version = 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] (/home/xxx/ansible-venv/bin/python3)
  jinja version = 3.1.3
  libyaml = True

$ ansible-galaxy collection list
Collection     Version
-------------- -------
ansible.netcommon     6.1.1
ansible.posix         1.5.4
ansible.utils         3.0.0
community.general     8.6.0
juniper.device        1.0.4

$ pip freeze
ansible-core==2.16.6
ansible-pylibssh==1.1.0
jsnapy==1.3.7
junos-eznc==2.7.0
ncclient==0.6.15
netmiko==4.3.0
paramiko==3.4.0
...

OS / Environment

Model: qfx5120-48y-8c Junos: 22.2R3-S2.8

Summary

the module always performs a commit confirmed operation

Steps to reproduce

  tasks :
    - name: SET JUNOS CONFIG
      juniper.device.config:
        load: "set"
        format: "set"
        lines: 'set system login message "This is login message"'
        comment: "comment"

Actual results

On device, we can see : 

# commit confirmed will be rolled back in 30 minutes

the behavior is the same if we do not specify the confirm parameter (like here) or if we specify it with a value (ex: 5 minutes)

dineshbaburam91 commented 4 months ago

I can't reproduce the issue, it is working as expected. "commit confirmed" will get triggered only when you pass confirmed: .

https://ansible-juniper-collection.readthedocs.io/config.html

Also, could you run the playbook enabling netconf trace options on the device and collect the playbook execution log if you find the issue?

set system services netconf traceoptions file netconf.log
set system services netconf traceoptions file size 1g
set system services netconf traceoptions flag all
---
- name: Test juniper. device.config module
  hosts: all
  gather_facts: no
  collections:
    - juniper.device

  tasks:
#################
    - name: "Execute set configuration"
      config:
        load: "set"
        format: "set"
        lines: 'set system login message "Login"'
        comment: "comment"
      register: test1
      tags: [ test1 ]

    - name: Check TEST 1
      debug:
        var: test1

Output:

TASK [Check TEST 1] *************************************************************************************************************************************
ok: [local_connection_testcases] => {
    "test1": {
        "changed": true,
        "diff": {
            "prepared": "\n[edit system]\n+   login {\n+       message Login;\n+   }\n"
        },
        "diff_lines": [
            "",
            "[edit system]",
            "+   login {",
            "+       message Login;",
            "+   }"
        ],
        "failed": false,
        "msg": "Configuration has been: opened, loaded, checked, diffed, committed, closed."
    }
}

Device output:

[edit]
jnpr# show system login    
message Login;

[edit]
jnpr# 

[edit]
jnpr#
slefol commented 4 months ago 

TASK [Execute set configuration] *************************************************************************************************************************************************************
changed: [tor-it02-10d-1] => {"changed": true, "diff_lines": ["", "[edit system login]", "-   message \"This is login message\";", "+   message Login;"], "msg": "Configuration has been: opened, loaded, checked, diffed, committed, closed."}

TASK [Check TEST 1] **************************************************************************************************************************************************************************
ok: [tor-it02-10d-1] => {
    "test1": {
        "changed": true,
        "diff": {
            "prepared": "\n[edit system login]\n-   message \"This is login message\";\n+   message Login;\n"
        },
        "diff_lines": [
            "",
            "[edit system login]",
            "-   message \"This is login message\";",
            "+   message Login;"
        ],
        "failed": false,
        "msg": "Configuration has been: opened, loaded, checked, diffed, committed, closed."
    }
}

# show system login message
message Login;

# commit confirmed will be rolled back in 29 minutes
{master:0}

# run show log netconf.log
May  6 12:59:49 [NETCONF] Started tracing session: 56384
May  6 12:59:49 [NETCONF] - [56384] Outgoing: <!-- No zombies were killed during the creation of this user interface -->
May  6 12:59:49 [NETCONF] - [56384] Outgoing: <!-- user ansible, class j-AUTOMATE -->
May  6 12:59:49 [NETCONF] - [56384] Outgoing: <hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
May  6 12:59:49 [NETCONF] - [56384] Outgoing:   <capabilities>
    <capability>urn:ietf:params:netconf:base:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:candidate:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:validate:1.0</capability>
    <capability>urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file</capability>
    <capability>urn:ietf:params:xml:ns:netconf:base:1.0</capability>
    <capability>urn:ietf:params:xml:ns:netconf:capability:candidate:1.0</capability>
    <capability>urn:ietf:params:xml:ns:netconf:capability:confirmed-commit:1.0</capability>
    <capability>urn:ietf:params:xml:ns:netconf:capability:validate:1.0</capability>
    <capability>urn:ietf:params:xml:ns:netconf:capability:url:1.0?scheme=http,ftp,file</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-yang-metadata?module=ietf-yang-metadata&amp;revision=2016-08-05</capability>
    <capability>urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring</capability>
    <capability>http://xml.juniper.net/netconf/junos/1.0</capability>
    <capability>http://xml.juniper.net/dmi/system/1.0</capability>
May  6 12:59:49 [NETCONF] - [56384] Outgoing:     <capability>http://yang.juniper.net/junos/jcmd?module=junos-configuration-metadata&amp;revision=2021-09-01</capability>
May  6 12:59:49 [NETCONF] - [56384] Outgoing:   </capabilities>
May  6 12:59:49 [NETCONF] - [56384] Outgoing:   <session-id>56384</session-id>
May  6 12:59:49 [NETCONF] - [56384] Outgoing: </hello>
May  6 12:59:49 [NETCONF] - [56384] Outgoing: ]]>]]>
May  6 12:59:49 [NETCONF] - [56384] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"><nc:capabilities><nc:capability>urn:ietf:params:netconf:base:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:base:1.1</nc:capability><nc:capability>urn:ietf:params:netconf:capability:writable-running:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:candidate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:confirmed-commit:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:startup:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:url:1.0?scheme=http,ftp,file,https,sftp</nc:capability><nc:capability>urn:ietf:params:netconf:capability:validate:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:xpath:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:notification:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:interleave:1.0</nc:capability><nc:capability>urn:ietf:params:netconf:capability:with-defaults:1.0</nc:capability></nc:capabilities></nc:hello>]]>]]>
May  6 12:59:49 [NETCONF] - [56384] Incoming: Received ']]>]]>' token which is split in two consecutive streams or has leading/trailing spaces/tabs or both.
May  6 12:59:49 [NETCONF] - [56384] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:804fcec7-a897-4a1e-b6e5-d31f9cff6d11"><lock-configuration/></nc:rpc>]]>]]>
May  6 12:59:49 [NETCONF] - [56384] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/22.2R0/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:804fcec7-a897-4a1e-b6e5-d31f9cff6d11">
May  6 12:59:49 [NETCONF] - [56384] Outgoing: </rpc-reply>
May  6 12:59:49 [NETCONF] - [56384] Outgoing: ]]>]]>
May  6 12:59:49 [NETCONF] - [56384] Incoming: Received ']]>]]>' token which is split in two consecutive streams or has leading/trailing spaces/tabs or both.
May  6 12:59:49 [NETCONF] - [56384] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:c15a4e51-b54b-4137-9c4f-18a2f5b4c643"><load-configuration format="text" action="set"><configuration-set>set system login message "Login"</configuration-set></load-configuration></nc:rpc>]]>]]>
May  6 12:59:49 [NETCONF] - [56384] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/22.2R0/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:c15a4e51-b54b-4137-9c4f-18a2f5b4c643">
May  6 12:59:49 [NETCONF] - [56384] Outgoing: <load-configuration-results>
May  6 12:59:49 [NETCONF] - [56384] Outgoing: <ok/>
May  6 12:59:49 [NETCONF] - [56384] Outgoing: </load-configuration-results>
May  6 12:59:49 [NETCONF] - [56384] Outgoing: </rpc-reply>
May  6 12:59:49 [NETCONF] - [56384] Outgoing: ]]>]]>
May  6 12:59:49 [NETCONF] - [56384] Incoming: Received ']]>]]>' token which is split in two consecutive streams or has leading/trailing spaces/tabs or both.
May  6 12:59:49 [NETCONF] - [56384] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f74f63ae-0418-4e3c-a123-baba46c2c32c"><commit-configuration><check/></commit-configuration></nc:rpc>]]>]]>
May  6 12:59:49 [NETCONF] - [56384] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/22.2R0/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f74f63ae-0418-4e3c-a123-baba46c2c32c">
May  6 12:59:49 [NETCONF] - [56384] Outgoing: <commit-results>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: <routing-engine junos:style="normal">
May  6 12:59:50 [NETCONF] - [56384] Outgoing: <name>localre</name>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: <commit-check-success/>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: </routing-engine>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: </commit-results>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: <ok/>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: </rpc-reply>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: ]]>]]>
May  6 12:59:50 [NETCONF] - [56384] Incoming: Received ']]>]]>' token which is split in two consecutive streams or has leading/trailing spaces/tabs or both.
May  6 12:59:50 [NETCONF] - [56384] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:3541cb6c-17e4-48f0-8b80-30710674056b"><get-configuration compare="rollback" rollback="0" format="text"/></nc:rpc>]]>]]>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/22.2R0/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:3541cb6c-17e4-48f0-8b80-30710674056b">
May  6 12:59:50 [NETCONF] - [56384] Outgoing: <configuration-information>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: <configuration-output>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: [edit system login]
May  6 12:59:50 [NETCONF] - [56384] Outgoing: -   message "This is login message";
May  6 12:59:50 [NETCONF] - [56384] Outgoing: +   message Login;
May  6 12:59:50 [NETCONF] - [56384] Outgoing: </configuration-output>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: </configuration-information>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: </rpc-reply>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: ]]>]]>
May  6 12:59:50 [NETCONF] - [56384] Incoming: Received ']]>]]>' token which is split in two consecutive streams or has leading/trailing spaces/tabs or both.
May  6 12:59:50 [NETCONF] - [56384] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:078eae3a-5c55-444c-8b65-8dcadeba44b3"><commit-configuration><log>comment</log><confirmed/><confirm-timeout>30</confirm-timeout></commit-configuration></nc:rpc>]]>]]>
May  6 12:59:50 [NETCONF] - [56384] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/22.2R0/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:078eae3a-5c55-444c-8b65-8dcadeba44b3">
May  6 12:59:50 [NETCONF] - [56384] Outgoing: <commit-results>
May  6 12:59:51 [NETCONF] - [56384] Outgoing: <routing-engine junos:style="normal">
May  6 12:59:51 [NETCONF] - [56384] Outgoing: <name>localre</name>
May  6 12:59:51 [NETCONF] - [56384] Outgoing: <commit-check-success/>
May  6 12:59:51 [NETCONF] - [56384] Outgoing: <commit-success/>
May  6 12:59:51 [NETCONF] - [56384] Outgoing: <commit-revision-information>
May  6 12:59:51 [NETCONF] - [56384] Outgoing: <new-db-revision>localre-1714993190-88</new-db-revision>
May  6 12:59:51 [NETCONF] - [56384] Outgoing: <old-db-revision>localre-1714993104-87</old-db-revision>
May  6 12:59:51 [NETCONF] - [56384] Outgoing: </commit-revision-information>
May  6 12:59:51 [NETCONF] - [56384] Outgoing: </routing-engine>
May  6 12:59:51 [NETCONF] - [56384] Outgoing: </commit-results>
May  6 12:59:51 [NETCONF] - [56384] Outgoing: <ok/>
May  6 12:59:51 [NETCONF] - [56384] Outgoing: </rpc-reply>
May  6 12:59:51 [NETCONF] - [56384] Outgoing: ]]>]]>
May  6 12:59:51 [NETCONF] - [56384] Incoming: Received ']]>]]>' token which is split in two consecutive streams or has leading/trailing spaces/tabs or both.
May  6 12:59:52 [NETCONF] - [56384] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f60e2530-c98d-4d46-8e45-f30c9cc81097"><unlock-configuration/></nc:rpc>]]>]]>
May  6 12:59:52 [NETCONF] - [56384] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/22.2R0/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:f60e2530-c98d-4d46-8e45-f30c9cc81097">
May  6 12:59:52 [NETCONF] - [56384] Outgoing: </rpc-reply>
May  6 12:59:52 [NETCONF] - [56384] Outgoing: ]]>]]>
May  6 12:59:52 [NETCONF] - [56384] Incoming: Received ']]>]]>' token which is split in two consecutive streams or has leading/trailing spaces/tabs or both.
May  6 12:59:52 [NETCONF] - [56384] Incoming: <?xml version="1.0" encoding="UTF-8"?><nc:rpc xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:38533529-7b13-4b29-a53a-be339957e8d6"><nc:close-session/></nc:rpc>]]>]]>
May  6 12:59:52 [NETCONF] - [56384] Outgoing: <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns:junos="http://xml.juniper.net/junos/22.2R0/junos" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="urn:uuid:38533529-7b13-4b29-a53a-be339957e8d6">
May  6 12:59:52 [NETCONF] - [56384] Outgoing: <ok/>
May  6 12:59:52 [NETCONF] - [56384] Outgoing: </rpc-reply>
May  6 12:59:52 [NETCONF] - [56384] Outgoing: ]]>]]>
May  6 12:59:52 [NETCONF] - [56384] Outgoing: <!-- session end at 2024-05-06 12:59:52 CEST -->
slefol commented 4 months ago

I noticed something strange: it looks like 2 commits are executed

show log messages
May  6 12:59:49  tor-it02-10d-1 mgd[56384]: UI_COMMIT: User 'ansible' requested 'commit' operation (comment: none)
May  6 12:59:50  tor-it02-10d-1 mgd[56384]: UI_COMMIT: User 'ansible' requested 'commit' operation (comment: comment)
May  6 12:59:51  tor-it02-10d-1 mgd[56384]: UI_COMMIT_CONFIRMED_REMINDER: 'commit confirmed' must be confirmed within 30 minutes
May  6 12:59:51  tor-it02-10d-1 mgd[56384]: UI_COMMIT_COMPLETED:  : commit complete
dineshbaburam91 commented 4 months ago

Are you seeing the same behavior when you try manually on the device?

slefol commented 4 months ago

manually on the device, the behavior is as expected :

user: ansible (with class AUTOMATE : permissions all; allow-commands "configure private"; deny-commands configure;)

ansible@tor-it02-10d-1> show configuration system login message
message Login;

ansible@tor-it02-10d-1> configure private
warning: uncommitted changes will be discarded on exit
Entering configuration mode
ansible@tor-it02-10d-1# set system login message "hello"
ansible@tor-it02-10d-1# commit confirmed 5
error: commit confirmed not supported for private configuration
ansible@tor-it02-10d-1# commit comment test
configuration check succeeds
commit complete

another user (permissions all; allow-commands "(configure private)|(configure exclusive)"; deny-commands configure;)

slefol@tor-it02-10d-1> show configuration system login message
message hello;
slefol@tor-it02-10d-1> configure exclusive
warning: uncommitted changes will be discarded on exit
Entering configuration mode
slefol@tor-it02-10d-1# set system login message "Login"
slefol@tor-it02-10d-1# commit confirmed 42
configuration check succeeds
commit confirmed will be automatically rolled back in 42 minutes unless confirmed
commit complete
# commit confirmed will be rolled back in 42 minutes
dineshbaburam91 commented 4 months ago

@slefol Could you please let me know the use case? Why are performing commit confirmed as well in manual execution?

slefol commented 4 months ago

Hi, @dineshbaburam91 I want to execute a simple commit without commit confirmed. it does not work. so I tried a confirmed commit but with 5 minutes, that doesn't work either (it's always 30 minutes whatever the value of the confirmed parameter). The only way I found to work around the problem while waiting for this bug to be resolved is to do a second task which confirms the commit with a commit check. I would have preferred to do it with a commit but it doesn't work and it's a shame because compared to a check commit we have a trace in the "show system commit" for confirmation which is not the case with a simple commit check.

slefol commented 4 months ago

To exclude a user profile problem, I created on the remote device a user without a profile (class super-user) and I used this user in the playbook. The result is the same. For information, a confirmed commit executed on the equipment rolls back for 10 minutes by default. I don't know why the equipment rolls back for 30 minutes using the juniper.device.config module. In the juniper/device/plugins/connection/pyez.py file, we have confirmed=None and in the playbook execution, we can see:

    "invocation": {
        "module_args": {
            "confirmed": null,
slefol commented 4 months ago

I looked through the module code and I think I identified the bug.

In file https://github.com/Juniper/ansible-junos-stdlib/blob/master/ansible_collections/juniper/device/plugins/module_utils/juniper_junos_common.py

The function name is commit_configuration.

The start of the function is :

    def commit_configuration(self, ignore_warning=None, comment=None,
                             confirmed=None, timeout=30, full=False,
                             sync=False, force_sync=False):

The commit is called later in the function 

        if self.conn_type != "local":
            self._pyez_conn.commit_configuration(ignore_warning, comment, timeout, confirmed, full, sync, force_sync)
            return

the order of the parameters is not right.

I was able to do a quick update to the juniper_junos_common.py and it seems to resolve the issue. I modified (by reversing the comment and confirmed parameters) the following code to the commit_configuration function:

        if self.conn_type != "local":
            self._pyez_conn.commit_configuration(ignore_warning, comment, confirmed, timeout, full, sync, force_sync)
            return

the playbook works as expected with a simple commit or a confirmed commit with confirmed: 42 for example

dineshbaburam91 commented 4 months ago

Thank you for investigating. I will review and resolve the issue.

dineshbaburam91 commented 4 months ago

Fixed #661

slefol commented 3 months ago

HI, @dineshbaburam91 there is an error in the proposed fix. see #661

dineshbaburam91 commented 3 months ago

Hi @slefol I have fixed and push the change