405 Not supported for system_global OS7.2.3

[lewis-green]

When doing a task with fortios_system_global I get a 405 method not supported. (FortiOS 7.2.3)

    - name: Set System Global
      fortios_system_global:
        access_token: "{{ fortinet_access_token }}"
        enable_log: true
        vdom: "{{ vdom }}"
        system_global:
          hostname: 'test'

In the log it says POST is not allowed. I took the call and did it in Postman with a PUT and it works.

It looks like the REST Method is wrong?

[lewis-green]

[ITRON-kpalmberg]

I believe I might be having a similar issue with fortios_system_vdom_exception. I'm getting some 405 errors in my logs even though the configuration we are setting hasn't been touched in 10 months. We're also running 7.2.3 of FortiOS.

I was previously running version 2.1.2 of the Ansible module and also just tried 2.2.2 which I believe is the latest. Same problems are occurring. It's making a POST to /api/v2/cmdb/system/vdom-exception/1 but then I get the error object already exists (mkey='1') in the httpsd logs. Technically the error makes sense, since the object does exist. The question is why is a POST request being made here instead of a PUT?

Similar to @lewis-green, I could also test a POST request in Postman (which failed with a 405) but then afterwards try a PUT request which succeeded with a 200.

I'm perplexed here what is causing the problem as we haven't changed our Ansible module version in many months. The only thing that has updated is the FortiOS version itself. But even considering that we still had some successful Ansible releases on 7.2.3 after the upgrading to this FortiOS version. I checked the known issues on 7.2.3 and don't see anything standing out that would impact this. I might poke around in the source code on this repo to see if anything stands out or look at the patch notes for 7.2.4. Possibly we can try to see if 7.2.4 fixes any issues if there's some problem with 7.2.3.

[lewis-green]

@ITRON-kpalmberg I decided to run a HTTPS traffic sniffer (reverse proxy and log the requests) and found the log saying method PUT but it actually sent a POST.

I am a bit confused as drilling down the HttpApi code and then Ansible's HttpApiBase code I can't see how a PUT turns in to a POST anywhere.

[ITRON-kpalmberg]

@lewis-green I'm still testing and doing some validation, but I appear to have been able to fix the issue by updating the Ansible version used for the deployment. I had this statically set to 2.10.7 to ensure stability of releases rather than using "latest" or something that could use a different version for different releases. Since I had a static version set I'm not sure how/why we would start seeing new issues on this version we've been using for a while. However, I tried 4.10.0, which appears to be the newest version I can use with my current Python version, and my Ansible deployments are working fine.

You might take a look at your Ansible version and consider upgrading if it's looking out of date.

[ITRON-kpalmberg]

@lewis-green I might have spoke too soon on the above comment...still having issues on 4.10.0.

[ITRON-kpalmberg]

I tried upgrading a FortiGate with this issue to 7.2.4. Afterwards, I started receiving 403s. I believe it's related to this issue that's also currently open. I supplied API credentials using access_token instead of the previous method as suggested in the comments on that issue here. This resolved my 403 auth issue after the upgrade, but now it seems I'm back to square one with the POST/PUT issue.

In the httpsd logs I see results like this:

[httpsd    info] fweb_debug_init[416] -- New POST request for "/api/v2/cmdb/system/vdom-exception/1" from "X.X.X.X:36714"
[httpsd    info] fweb_debug_init[418] -- User-Agent: "Python-urllib/3.6"
[httpsd    info] fweb_debug_init[420] -- Handler "api_cmdb_v2-handler" assigned to request
[httpsd warning] api_access_check_for_api_key[688] -- API Key request authorized for admin_user_name_placeholder from X.X.X.X.
[httpsd    info] api_store_parameter[320] -- add API parameter 'vdom' (type=string)
[httpsd    info] api_store_parameter[320] -- add API parameter 'access_token' (type=string)
[httpsd    info] api_store_parameter[320] -- add API parameter 'id' (type=int)
[httpsd    info] api_store_parameter[320] -- add API parameter 'object' (type=string)
[httpsd    info] api_cmdb_request_init_by_path[1800] -- new CMDB query (path='system',name='vdom-exception')
[httpsd    info] api_cmdb_request_init_by_path[1829] -- querying CMDB entry (mkey='1')
[httpsd    info] handle_cli_req_v2[3200] -- no method found for requested action: (null)
[httpsd    info] handle_cli_req_v2[3318] -- returning to original vdom "root"
[httpsd warning] api_return_http_result[1272] -- API error 405 raised
[httpsd    info] fweb_debug_final[306] -- Completed POST request for "/api/v2/cmdb/system/vdom-exception/1" (HTTP 405)
[httpsd    info] fweb_debug_init[416] -- New POST request for "/api/v2/cmdb/system/vdom-exception" from "X.X.X.X:36716"
[httpsd    info] fweb_debug_init[418] -- User-Agent: "Python-urllib/3.6"
[httpsd    info] fweb_debug_init[420] -- Handler "api_cmdb_v2-handler" assigned to request
[httpsd warning] api_access_check_for_api_key[688] -- API Key request authorized for admin_user_name_placeholder from X.X.X.X.
[httpsd    info] api_store_parameter[320] -- add API parameter 'vdom' (type=string)
[httpsd    info] api_store_parameter[320] -- add API parameter 'access_token' (type=string)
[httpsd    info] api_store_parameter[320] -- add API parameter 'id' (type=int)
[httpsd    info] api_store_parameter[320] -- add API parameter 'object' (type=string)
[httpsd    info] api_cmdb_request_init_by_path[1800] -- new CMDB query (path='system',name='vdom-exception')
[httpsd    info] handle_cli_req_v2[3288] -- new CMDB API request (vdom='root',user='admin_user_name_placeholder')
[httpsd    info] _api_cmdb_v2_config[1419] -- editing CLI object (append=1, auto_key=1, path=system, name=vdom-exception, mkey=(null), flags=0)
[httpsd   error] cmdb_commit_from_json[2096] -- object already exists (mkey='1')
[httpsd   error] _api_cmdb_v2_config[1455] -- error editing object (nret=-5)
[httpsd warning] api_return_http_result[1272] -- API error -5 raised
[httpsd    info] handle_cli_req_v2[3318] -- returning to original vdom "root"
[httpsd    info] fweb_debug_final[306] -- Completed POST request for "/api/v2/cmdb/system/vdom-exception" (HTTP 500)

It appears there's multiple POST requests in here. One results in a 405 and the other results in a 500.

FortiOS version: 7.2.4 Ansible version: 4.10.0 FortiOS module version: 2.2.2

[ITRON-kpalmberg]

If I delete the object in the FortiGate then the Ansible deployment successfully creates the resource. Running the deployment a 2nd time without any changes fails due to the above mentioned issues. It seems somehow this is not operating in an idempotent manner right now.

[lewis-green]

Ah yes I did spot that for the api user I made in the playbook. But I then had to do a work around to say if the step has run skip it. Which as you say is not right.

It's a bit hard to delete the system global config since that's always there :)

[kichitan]

Same issue over here:

ansible [core 2.14.3] Ubuntu 22.04.1 LTS fortinet.fortios-2.2.2.info FortiGate 7.2.4

[ITRON-kpalmberg]

@kichitan what's your FortiOS version? Could be useful info.

[lewis-green]

@ITRON-kpalmberg After a bit of digging I think I've found the issue may be Ansible's HTTP API Connection class. The FortiOS Collection passes method=PUT but as far as I can see it never gets set on the connection.

I will dig further

[lewis-green]

@ITRON-kpalmberg Found the issue. This PR fixes it https://github.com/ansible-collections/ansible.netcommon/pull/523

[ITRON-kpalmberg]

@lewis-green thanks for the details, I hope that can be the resolution in my case too. I will investigate the PR a bit. I'm assuming some version upgrade for Ansible needs to be applied to fix the bug. Hopefully, there's a version out that has this patch, as I can see the PR was only completed in the past week.

[lewis-green]

@ITRON-kpalmberg the latest released collection for netcommon to galaxy has the bug :(.

I patched my local copy of the collection with that patch and all my playbooks work happily now.

[ITRON-kpalmberg]

@lewis-green ugh, thanks for the tip. I'm going to give this a try myself by patching the local copy. It's just going to add extra complications to existing pipeline automation.

[ITRON-kpalmberg]

@lewis-green possibly using an older version of ansible.netcommon might be a solution too. It looks like the re-added code might have been removed in this PR that's tagged with v5.0.0. I'm going to try an older version of ansible.netcommon prior to this PR to see if just specifying an earlier version before v5.0.0 might work.

[ITRON-kpalmberg]

@lewis-green since as you noted it seems the issue is impacting 5.0.0, I tried forcing my agent for a pipeline run to use 4.1.0, which is the most recent version of ansible.netcommon prior to 5.0.0. Version 4.1.0 was releases 4 months ago, which was prior to the recent pull requests on the ansible.netcommon repo that introduced this bug. So far after forcing the use of 4.1.0 my pipeline is running successfully in my dev environment!

[ITRON-kpalmberg]

For now we have a couple viable work arounds that seem to be fixing issues so far discussed in this thread, due to an issue with the underlying ansible.netcommon code:

• Force the use of version 4.1.0 of ansible.netcommon which is a version prior to when the bug was introduced.
• If using the latest 5.0.0, you can make local modifications to the package as described by @lewis-green to include the bug fix, as there isn't a version of ansible.netcommon released yet that contains the bug fix.

[ITRON-kpalmberg]

@kichitan hope the comments above can help resolve your issue as well.

[kichitan]

Hi @ITRON-kpalmberg I'll give it a try, but for the looks of it. I think this will just do it. Thank you guys :)

[MaxxLiu22]

Hi all,

Thank you @lewis-green for bring up this issue, and thank you all for your debugging effort. I can reproduce this issue with ansible.netcommon:5.0.0, I will report it to the development team to see if they can fix this issue with latest ansible.netcommon version, for now we suggest to downgrade ansible.netcommon to version 4.1.0 by running ansible-galaxy collection install ansible.netcommon:4.1.0. Thank you all again for providing these valuable information.

Thanks, Maxx

[kichitan]

@MaxxLiu22 I tried that but still not able to make work a basic playbook against the fortigate. The method change to put but now I get a 403.

/home/ubuntu/.local/lib/python3.10/site-packages/ansible_collections

Collection Version

---

amazon.aws 5.2.0
ansible.netcommon 4.1.0

fatal: [fortigate01]: FAILED! => {"changed": false, "meta": {"build": 1396, "http_method": "PUT", "http_status": 403, "name": "global", "path": "system", "serial": "xxxxxx", "status": "error", "vdom": "root", "version": "v7.2.4"}, "msg": "Error in repo"}

[ITRON-kpalmberg]

@kichitan see my comment here earlier in this thread. The 403 issue appears to be related to other aspects specifically in 7.2.4. I tested updating a FortiGate from 7.2.3 to 7.2.4 and I received the 403 issues as soon as I updated to 7.2.4. I did not have the issues in 7.2.3. You can see my comment for other details, but I fixed the issue by providing access_token to tasks instead of alternate methods like ansible_ssh_pass.

[MaxxLiu22]

@kichitan yes just like @ITRON-kpalmberg said, we suggest use token to access FGT, for how to generate and use a token you can take this as a reference, if you still have any questions, feel free to ask me.

Thanks, Maxx

[kichitan]

@MaxxLiu22 @ITRON-kpalmberg Thank you for your input guys, I test the token workaround and it works for me on version 7.2.4 and the username and password on version 7.2.3 and it also works there :).