yamabay
commented
6 months ago Complex Test Case
I wanted to provide better context of the exact code I am trying to implement to assist in determining if this is a bug or simply bad code I am writing!
I am using a YAML file that define the ACL. The YAML file is converted to a Terraform object using the yamldecode function. Once this is done, a for loop is used within the entries argument of the extended ACL resource to loop through each statement within the YAML file for the ACL.
An example YAML file looks like this:
acls:
YAML_ACL:
- action: "permit"
protocol: "udp"
source: "any"
destination: "1.1.1.1/32"
dest_port: "53"
- action: "permit"
protocol: "udp"
source: "any"
destination: "1.1.1.2/32"
dest_port: "53"
- action: "permit"
protocol: "udp"
source: "any"
destination: "1.1.1.3/32"
dest_port: "53"Within the locals block, Terraform will render the YAML file:
locals {
acls = yamldecode(file("acls.yaml"))
}The statements within the YAML config are then looped through within the entries attribute as shown in this resource:
resource "iosxe_access_list_extended" "test" {
name = "bug-test-complex"
entries = [
for rule in local.acls["acls"]["YAML_ACL"]:
{
sequence = index(local.acls["acls"]["YAML_ACL"], rule) + 1
ace_rule_action = rule.action
ace_rule_protocol = rule.protocol
source_any = rule.source == "any" ? true : null
source_prefix = rule.source == "any" ? null : element(split("/", rule.source), 0)
source_prefix_mask = rule.source == "any" ? null : local.prefix_masks[element(split("/", rule.source), 1)]
source_port_equal = lookup(rule, "source_port", null)
destination_any = rule.destination == "any" ? true : null
destination_prefix = rule.destination == "any" ? null : element(split("/", rule.destination), 0)
destination_prefix_mask = rule.destination == "any" ? null : local.prefix_masks[element(split("/", rule.destination), 1)]
destination_port_equal = lookup(rule, "dest_port", null)
}
]
}Note: Not shown for brevity, but there is a local.prefix_masks map that will convert a subnet CIDR notation to its associated wildcard mask.
Once this Terraform code is applied, the router will have an initial ACL of:
bug-test-rtr#show ip access-list | s bug-test-complex
Extended IP access list bug-test-complex
1 permit udp any host 1.1.1.1 eq domain
2 permit udp any host 1.1.1.2 eq domain
3 permit udp any host 1.1.1.3 eq domainIf an ACL entry is added to the YAML configuration, this will work as expected. Here is the new YAML structure to show the rule change:
acls:
YAML_ACL:
- action: "permit"
protocol: "udp"
source: "any"
destination: "1.1.1.1/32"
dest_port: "53"
- action: "permit"
protocol: "udp"
source: "any"
destination: "1.1.1.2/32"
dest_port: "53"
- action: "permit"
protocol: "udp"
source: "any"
destination: "8.8.8.8/32"
dest_port: "53"
- action: "permit"
protocol: "udp"
source: "any"
destination: "1.1.1.3/32"
dest_port: "53"Note: A new entry was added as the third entry in the list.
This is accurately reflected on the router once the Terraform code is applied:
bug-test-rtr#show ip access-list | s bug-test-complex
Extended IP access list bug-test-complex
1 permit udp any host 1.1.1.1 eq domain
2 permit udp any host 1.1.1.2 eq domain
3 permit udp any host 8.8.8.8 eq domain
4 permit udp any host 1.1.1.3 eq domainThe bug starts once that entry is removed (or any other entry). If the new entry is removed from the YAML configuration and the Terraform code is applied, the router will now show the ACL as having only two entries:
bug-test-rtr#show ip access-list | s bug-test-complex
Extended IP access list bug-test-complex
1 permit udp any host 1.1.1.1 eq domain
2 permit udp any host 1.1.1.2 eq domainThe current fix for this is to completely destroy the ACL before recreating it, but the provider does not currently recreate the resource if a change to the entries attribute is detected, so this is done by removing the ACL from the configs before recreating it.
I am trying to better understand if this issue is related to the provider, the underlying YANG module, or other Terraform behavior that may indicate this is expected behavior.
Thank you for any assistance you can provide with this!
danischm
Overview
The iosxe_access_list_extended resource does not configure an ACL correctly once modifications to the ACL sequence numbers are made. When the sequence of statements are reordered, Terraform reflect these changes in the Terraform plan. However, once the changes are applied, the ACL will be missing statements. Subsequent runs of
terraform planwill not detect this drift and will report that no changes need to made. Terraform believes the live configurations are accurately matching the declarative state which is not accurate.A
terraform plan -refresh-onlyalso does not detect the mismatch between the live configurations and the state.Test Case
This bug can be replicated by following the steps list below.
Once applied, the
bug-test-aclACL should now exist on the router:Note: The sequence number for the second and third statement have been swapped.
When the
terraform planis ran, the only change detected is the sequence number. However, these statements have different configurations so the entire statement should be recognized as a change.Here is an example output of the
terraform plan:Once this is applied, the live configuration of the ACL on the router looks like this:
The debug of the
terraform applyshows the payload of the PATCH has the appropriate configurations that reflect the desired configurations in the resource:Note: The resource can be removed and recreated without any issues even though the internal configurations do not match the configurations in the state.
Other Scenarios
I have tested adding and removing statements to an ACL and that appears to work successfully. The bug appears to occur only when sequences are changes for existing statements.