panos_security_rule_group this resource doesn't delete the rules from the rules group during the configureRule and deleterule

[crkg]

Describe the bug

The deletion of multiple rules is failing from the PANOS provider, when provided in a group to create/update or delete rules. the API is throwing an Error: The request could not be handled .

Expected behavior

When the rules are provided to the panos_security_rule_group the provider should be capable to handle the deletion of multiple rules at the same time, it seems the deletion is not working as expected when passed in the group.

Current behavior

The deletion of multiple rules is failing from the PANOS provider, when provided in a group to create/update or delete rules. the API is throwing an Error: The request could not be handled .

Possible solution

A PR is raised for the same - https://github.com/PaloAltoNetworks/pango/pull/92/files ; allow deletion of rules when provided as group create/update/delete together.

Steps to reproduce

1.Create a rule group with "panos_security_rule_group" with multiple rules(3 or more) in it 2.Attempt to delete the entire rule group with terraform destroy 3.Try removing multiple rules(2 or more) from the rule group input and then terraform apply

As Sohel described below, and according to the below screen shot you see the request could not be handled. it seems the endpoint in the PANOS doesnt seems to accept deletion/modification. some message is masked in the image due to sensitive data. how ever they are just encoded string with rules collection.

Screenshots

Context

Your Environment

• Version used:

• panos = { source = "paloaltonetworks/panos" version = "~> 1.9.0" }

• Environment name and version (e.g. Chrome 59, node.js 5.4, python 3.7.3):

• Operating System and version (desktop or mobile):

• Link to your project:

[welcome-to-palo-alto-networks[bot]]

:tada: Thanks for opening your first issue here! Welcome to the community!

[shinmog]

OK, so surfacing the error instead of hiding it is one thing. Can you tell me a bit more about why this error is happening / a way to reproduce it / what steps you had to take to actually delete the error..?

[sohel-m]

Hello @shinmog

Allow me to elaborate on the issue my colleague has raised above.

Firstly the "environment"

Terraform panos provider version: 1.9.2 ( similar behaviour also observed with 1.10.3 i.e latest version) PAN OS version on the Panorama: 10.1.6-h3

Reproducing the error: Create a rule group with "panos_security_rule_group" with multiple rules(3 or more) in it. 1)Attempt to delete the entire rule group with terraform destroy 2)Try removing multiple rules(2 or more) from the rule group input and then terraform apply

Issues : In (1) above, you will see "The request could not be handled" error message as seen from the ss above. In (2) the terraform apply wrongly succeeds because of error suppression. I think point (2) can be fixed by https://github.com/PaloAltoNetworks/pango/pull/92/files But after it is fixed it will still run into (1) again :)

The culprit I believe is the PAN OS itself. It is not able to handle the delete operation with multiple rules at once; specifically the operations which generate the 'FULL PATH' like below in Configuration logs under MONITOR tab in Panorama

/config/devices/entry[@name='localhost.localdomain']/device-group/entry[@name='<device group name>']/pre-rulebase/security/rules/entry[@name='my_rule_to_delete_1' or @name='my_rule_to_delete_2' or @name='my_rule_to_delete_3']

Upon some investigation of logs on our Panorama(10.1.6-h3), I see all operations like above FAIL. It also fails with Panorama(10.1.5-h1).

During development the code was tested against Panorama(10.1.3) and the operations like above were successful.

So I believe the upgrade might be having a bug. In any case this is my reasoning from the limited investigation on my side.

Could you please confirm on your side and come up with a solution?

Thanks

[crkg]

The culprit is the PANOS library, in order to replicate the error you can follow the exact same steps to reproduce. please limit the version to 1.9.2 or latest. it seems to us when multiple elements passed to the end point to perform CURD there is an issue which is not tested from the release. let us know if you need any more information.

[crkg]

@shinmog , do you have any updates with regards to the above issue ?

[shinmog]

@crkg

Ok, so this issue is the same in nature to #354 .

I've tried three different PAN-OS versions. PAN-OS 10.1.3, PAN-OS 10.1.6-h6, and PAN-OS 10.2.2. Of those three, I can reproduce this issue only on 10.1.6-h6, which means that you've found a PAN-OS bug and need to reach out to TAC. This issue was introduced somewhere between 10.1.4 and 10.1.6-h3, since that's what you're running, and still exists in 10.1.6-h6.

When working with TAC, since Terraform is community supported you'll want to use the new logging options the provider has present to log commands as cURL commands, as TAC has to support cURL commands directly against the XML API. You can find the docs on that here.

As for the provider, I'll go ahead and surface the error so users at least know what's going on, but as for resolving the issue, it seems like you need to get off of 10.1.6-h3.

[crkg]

@shinmog , Thanks for the summary. since there is a challenge to upgrade to 10.2.x panorama. do you think its feasible to patch the fix to 10.1.6-x ?

[shinmog]

@crkg

Possibly..? But that's not our team's wheelhouse, unfortunately. We can only affect the provider itself on our end. Working through your account manager and TAC is probably the best way to get this solved from the PAN-OS side.

But this is a regression that they caused, so they need to fix the regression they introduced.

[crkg]

@shinmog

Thanks for your support.

This is to let you know we have decided to upgrade our PAN OS version from 10.1.6-h3 to 10.2.2-h2. we will be updating them and validation the automation with provider version same as above 1.9.0.

please keep the issue open, if we will need anything to be updated. thanks again.

[shinmog]

@crkg

AFAIK 10.2.2 should be fine. Is multi-delete working ok for you?

[crkg]

Thanks for the support @shinmog .

yes, the batch delete is working as expected on 10.2.2 PanOS. We will like to monitor couple of weeks and confirm the change.

[crkg]

Hi @shinmog , you may mark this issue closed. thank you the issue is fixed with version 10.2.2-h2

[shinmog]

So, I will go ahead and close this out, but I've heard from a colleague that 11.0 has the same limitation with multi-delete that 10.1.5 does. I've not been able to personally verify this yet.

If this is the case, then upgrading from 10.2.2-h2 to 11.0 will have the same issue as 10.1.5 and onwards does.

Depending on how the internal issue is handled, the panos provider may need to be updated to accommodate this.