community.routeros.api_find_and_modify is not idempotent

[izzzhoga]

SUMMARY

Hello! I am using the community.routeros.api_find_and_modify module to configure the snmp community. When the role is first started, the default community will change to the required one, at the end the host state is "changed". When restarting and the configured community, the task state is "changed", although old_data and new_data are the same.

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.routeros.api_find_и_modify

ANSIBLE VERSION

ansible [core 2.13.6]
ansible [core 2.13.6]
  config file = /home/user/mkt/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/dist-packages/ansible
  ansible collection location = /home/user/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110]
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION

# /usr/local/lib/python3.9/dist-packages/ansible_collections
Collection         Version
------------------ -------
community.routeros 2.4.0  

# /home/user/.ansible/collections/ansible_collections
Collection         Version
------------------ -------
community.routeros 2.4.0

OS / ENVIRONMENT

MikroTik RouterOS 6.46.5

STEPS TO REPRODUCE

- name: Configure SNMP Community and other
  community.routeros.api_find_and_modify:
    path: snmp community
    find:
      default: yes
    values:
      addresses: "{{ ZP_Server }},{{ local_zb_proxy_ip }}"
      name: "{{ SNMP_COMMUNITY }}"
      security: none
      disabled: no
  register: snmp_community

- name: debug snmp_community
  debug:
    msg: "{{ snmp_community }}"

EXPECTED RESULTS

When the role is restarted, it is expected that the task state will be "changed=0"

ACTUAL RESULTS

Identical old_data and new_data, with the state "changed=1" at the end after the role is restarted

"new_data": [
    {
        ".id": "*0",
        "addresses": "0.0.0.0,1.1.1.1",
        "authentication-password": "",
        "authentication-protocol": "MD5",
        "default": true,
        "disabled": false,
        "encryption-password": "",
        "encryption-protocol": "DES",
        "name": "name",
        "read-access": true,
        "security": "none",
        "write-access": false
    }
],
"old_data": [
    {
        ".id": "*0",
        "addresses": "0.0.0.0,1.1.1.1",
        "authentication-password": "",
        "authentication-protocol": "MD5",
        "default": true,
        "disabled": false,
        "encryption-password": "",
        "encryption-protocol": "DES",
        "name": "name",
        "read-access": true,
        "security": "none",
        "write-access": false

[izzzhoga]

Why is this task not idempotent if the initial and final state of the system are the same?

[felixfontein]

I'll have to check out the sources. It definitely does look wrong...

Did you also run the playbook with --diff to see whether that prints something?

[izzzhoga]

I launched the playbook with the --diff flag, here is the output:

TASK [1DEV/mkt-config-snmp : Configure SNMP Community and other] **********************************************************
--- before
+++ after
@@ -2,7 +2,7 @@
     "values": [
         {
             ".id": "*0",
-            "addresses": "xx.xx.xx.xx/32,yy.yy.yy.yy/32",
+            "addresses": "xx.xx.xx.xx,yy.yy.yy.yy",
             "authentication-password": "",
             "authentication-protocol": "MD5",
             "default": true,

changed: [Mikrotik]

[felixfontein]

What does "{{ ZP_Server }},{{ local_zb_proxy_ip }}" evaluate to?

Also this diff does not match the output you have further up. There old_data has 0.0.0.0,1.1.1.1 for addresses, but according to the diff addresses in old_data should have been xx.xx.xx.xx/32,yy.yy.yy.yy/32.

[izzzhoga]

These are the ip addresses that I don't want to show :) But I can assure you that they are identical. As I understand it, the problem itself lies in the subnet mask: xx.xx.xx.xx/32.

(0.0.0.0,1.1.1.1 and xx.xx.xx.xx,yy.yy.yy.yy are the same)

[felixfontein]

But where is that mask? In the input data? In the data sent by the router?

[izzzhoga]

I think I was able to solve it on my own. A very stupid mistake. The addresses in Mikrotik look like this: But I passed them to Ansible without a subnet mask (without /32).

Then I tried to pass an ip address with a subnet mask in a variable, now everything works correctly. Maybe I'm not explaining it well, but I'm ready to tell you more if you have any questions.

I think the issue can be closed :)

[izzzhoga]

I have another question about the idempotency of the module, but another one. When using the community.routeros.api, I run the add command. When restarting the Playbook, this command is executed each time, adding the same entries to Mikrotik.

 - name: Configure radius rules in last number
    community.routeros.api:
      path: "radius"
      add: "address={{RADIUS_SERVER}} secret={{RADIUS_COMMON_SECRET}} service=login src-address={{main_ip_address}}"
    register: raduis_rules

Output after the first run of the playbook and after the second run:

ok: [Mikrotik] => {
    "msg": {
        "changed": true,
        "failed": false,
        "msg": [
            "added: .id= *1"
        ]
    }
}

ok: [Mikrotik] => {
    "msg": {
        "changed": true,
        "failed": false,
        "msg": [
            "added: .id= *2"
        ]
    }
}

Is this the normal behavior of the module? The essence of idempotence is violated, I already have such a rule, why add more?

[izzzhoga]

The same incomprehensible behavior of community.routeros.api with the add command is observed when adding an snmp community:

- name: Add customer Configure SNMP Community
  community.routeros.api:
    path: "snmp community"
    add: "addresses=0.0.0.0/0 name={{ CST_LOCAL_SNMP_SERVER }} security=none disabled=no"
  register: snmp_add

Output after the second run of the Playbook:

TASK [1DEV/mkt-config-snmp : Add customer Configure SNMP Community] **************************************************************************************
fatal: [Mikrotik]: FAILED! => {"changed": false, "msg": ["failure: community with the same name already exists!"]}

The same error is observed in Mikrotik itself:

Why is Ansible trying to add the snmp community again if it already exists?

[felixfontein]

I think I was able to solve it on my own. A very stupid mistake. The addresses in Mikrotik look like this: But I passed them to Ansible without a subnet mask (without /32).

Then I tried to pass an ip address with a subnet mask in a variable, now everything works correctly. Maybe I'm not explaining it well, but I'm ready to tell you more if you have any questions.

So basically this means that you didn't pass it in without a subnet mask to the module, but the API returned it with a subnet mask? The new_data / old_data you pasted did not have that subnet mask.

In any case, the module does not know which strings RouterOS treats to be equal, so if it sees a difference, it will update the record with what you passed into the module. So adjusting what you put in (as you now did) is the correct way. In any case, --diff is helpful here since it shows you what the module sees as differences.

[felixfontein]

I have another question about the idempotency of the module, but another one. When using the community.routeros.api, I run the add command. [...] Is this the normal behavior of the module? The essence of idempotence is violated, I already have such a rule, why add more?

The api module is as idempotent as the command module: you yourself are responsible of making its calls idempotent. If you want idempotency, use the api_find_and_modify or api_modify modules (if they support your use-case).

[izzzhoga]

I don't quite understand you... Please explain what logic is involved in the operation of the community.routeros.api module within this parameter?

Do I understand correctly that Ansible will always try to execute the add command regardless of the Mikrotik settings by the specified path?

[felixfontein]

Yes, it will always execute it. The api module has no way of knowing whether the add (or any other of the commands you use it to run) command is still necessary or not.

[izzzhoga]

Then how can the community.routeros.api module be idempotent if the add command is always executed?

An operation is considered idempotent if its repeated execution leads to the same result as a single execution.

[felixfontein]

The community.routeros.api module is never idempotent by itself. You have to use when: to only run it when needed.

[felixfontein]

The same for the command/shell/raw and community.routeros.command modules. You are responsible yourself to make your use of them idempotent.

[izzzhoga]

The api module is as idempotent as the command module

Apparently I misunderstood this comment. In any case, it became clearer to me, thank you for your help!

[izzzhoga]

Do I understand correctly from my practice that the modules community.routeros.api_find_and_modify module and community.routeros.api_modify module are idempotent?

[felixfontein]

Yes, they are. Assuming you pass in the data in the same format as the API returns it, like with the subnet mask in case of the IPs in the original post in this issue. The modules do not understand the API that well :)

[izzzhoga]

Great, thanks for the explanation and quick answers :)