ENG-12949: Policy engine providers

[gengdahlCyral]

Description of the change

Adding terraform provider for policy v2 API: /v2/policies that are planned for relase in v4.15.

The resource handles policy resource life-cycle (create/update/delete and import) The datasource handles read. There is no TF resource that handles the filtering API (ie where you can search using posting of predicates).

Type of change

• [ ] Bug fix (non-breaking change that fixes an issue)
• [x] New feature (non-breaking change that adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklists

Development

• [x] Lint rules pass locally
• [x] The code changed/added as part of this pull request has been covered with tests
• [x] All tests related to the changed code pass in development

Code review

• [ ] This pull request has a descriptive title and information useful to a reviewer. There may be a screenshot or screencast attached
• [ ] Jira issue referenced in commit message and/or PR title

Testing

Resource, creating three policies

terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # cyral_policy_v2.approval_policy_example will be created
  + resource "cyral_policy_v2" "approval_policy_example" {
      + created      = (known after apply)
      + description  = "Approval policy for emergency access"
      + document     = jsonencode(
            {
              + approvals  = {
                  + delete = {
                      + locations = [
                          + "gym_db.users",
                        ]
                      + tags      = [
                          + "PII",
                        ]
                    }
                  + read   = {
                      + locations = [
                          + "gym_db.users",
                        ]
                      + tags      = [
                          + "PII",
                        ]
                    }
                  + update = {
                      + locations = [
                          + "gym_db.users",
                        ]
                      + tags      = [
                          + "PII",
                        ]
                    }
                }
              + conditions = [
                  + {
                      + attribute = "role"
                      + operator  = "equals"
                      + value     = "admin"
                    },
                ]
              + identity   = {
                  + email = "gym_owner@example.com"
                }
            }
        )
      + enabled      = true
      + id           = (known after apply)
      + last_updated = (known after apply)
      + name         = "approval_policy"
      + tags         = [
          + "approval",
          + "emergency",
        ]
      + type         = "approval"
      + valid_from   = "2024-06-12T00:00:00Z"
      + valid_until  = "2024-06-13T00:00:00Z"

      + scope {
          + repo_ids = [
              + "gym_repo",
            ]
        }
    }

  # cyral_policy_v2.global_policy_example will be created
  + resource "cyral_policy_v2" "global_policy_example" {
      + created      = (known after apply)
      + description  = "Global policy for finance users with row limit for PII data"
      + document     = jsonencode(
            {
              + governedData = {
                  + tags = [
                      + "PII",
                    ]
                }
              + readRules    = [
                  + {
                      + conditions  = [
                          + {
                              + attribute = "identity.userGroups"
                              + operator  = "contains"
                              + value     = "finance"
                            },
                        ]
                      + constraints = {
                          + maxRows = 5
                        }
                    },
                  + {
                      + conditions  = []
                      + constraints = {}
                    },
                ]
            }
        )
      + enabled      = true
      + enforced     = true
      + id           = (known after apply)
      + last_updated = (known after apply)
      + name         = "global_policy"
      + tags         = [
          + "finance",
          + "global",
        ]
      + type         = "global"

      + scope {
          + repo_ids = [
              + "2gaWEAyeKbETyUy1LSx985gVqrk",
            ]
        }
    }

  # cyral_policy_v2.local_policy_example will be created
  + resource "cyral_policy_v2" "local_policy_example" {
      + created      = (known after apply)
      + description  = "Local policy to allow gym users to read their own data"
      + document     = jsonencode(
            {
              + governedData = {
                  + locations = [
                      + "gym_db.users",
                    ]
                }
              + readRules    = [
                  + {
                      + conditions  = [
                          + {
                              + attribute = "identity.userGroups"
                              + operator  = "contains"
                              + value     = "users"
                            },
                        ]
                      + constraints = {
                          + datasetRewrite = "SELECT * FROM ${dataset} WHERE email = '${identity.endUserEmail}'"
                        }
                    },
                ]
            }
        )
      + enabled      = true
      + enforced     = true
      + id           = (known after apply)
      + last_updated = (known after apply)
      + name         = "local_policy"
      + tags         = [
          + "gym",
          + "local",
        ]
      + type         = "local"

      + scope {
          + repo_ids = [
              + "2gaWEAyeKbETyUy1LSx985gVqrk",
            ]
        }
    }

Plan: 3 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

cyral_policy_v2.local_policy_example: Creating...
cyral_policy_v2.global_policy_example: Creating...
cyral_policy_v2.approval_policy_example: Creating...
cyral_policy_v2.local_policy_example: Creation complete after 2s [id=c11d51eb-48f6-47d2-b39d-fdf694060885]
cyral_policy_v2.global_policy_example: Creation complete after 2s [id=33aa02d2-2007-4dcf-b2dd-4841c7970a2f]
cyral_policy_v2.approval_policy_example: Creation complete after 2s [id=107ebed2-c54c-4c00-9e3e-9435a04fb179]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Datasource, read a policy

terraform apply
data.cyral_policy_v2.test_policy: Reading...
cyral_policy_v2.approval_policy_example: Refreshing state... [id=107ebed2-c54c-4c00-9e3e-9435a04fb179]
cyral_policy_v2.local_policy_example: Refreshing state... [id=c11d51eb-48f6-47d2-b39d-fdf694060885]
cyral_policy_v2.global_policy_example: Refreshing state... [id=33aa02d2-2007-4dcf-b2dd-4841c7970a2f]
data.cyral_policy_v2.test_policy: Read complete after 1s [id=504f0cb9-4556-4f90-9819-bfe4a96910db]

Changes to Outputs:
  + policy_description = "Local policies for users table access"
  + policy_document    = jsonencode(
        {
          + governedData = {
              + locations = [
                  + "gym_db.users",
                ]
            }
          + readRules    = [
              + {
                  + conditions  = [
                      + {
                          + attribute = "identity.userGroups"
                          + operator  = "contains"
                          + value     = "USERS"
                        },
                    ]
                  + constraints = {
                      + datasetRewrite = "SELECT * FROM ${dataset} WHERE email = '${identity.endUserEmail}'"
                    }
                },
              + {
                  + conditions  = []
                  + constraints = {}
                },
            ]
        }
    )
  + policy_name        = "policy1"

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

policy_description = "Local policies for users table access"
policy_document = "{\"governedData\":{\"locations\":[\"gym_db.users\"]},\"readRules\":[{\"conditions\":[{\"attribute\":\"identity.userGroups\",\"operator\":\"contains\",\"value\":\"USERS\"}],\"constraints\":{\"datasetRewrite\":\"SELECT * FROM ${dataset} WHERE email = '${identity.endUserEmail}'\"}},{\"conditions\":[],\"constraints\":{}}]}"
policy_name = "policy1"

resource, update policies

goran@goran-engdahl-ThinkPad-P1-Gen-6:~/work/terraform$ terraform apply
cyral_policy_v2.local_policy_example: Refreshing state... [id=c11d51eb-48f6-47d2-b39d-fdf694060885]
cyral_policy_v2.global_policy_example: Refreshing state... [id=33aa02d2-2007-4dcf-b2dd-4841c7970a2f]
cyral_policy_v2.approval_policy_example: Refreshing state... [id=107ebed2-c54c-4c00-9e3e-9435a04fb179]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # cyral_policy_v2.approval_policy_example will be updated in-place
  ~ resource "cyral_policy_v2" "approval_policy_example" {
      ~ description  = "Approval policy for emergency access" -> "Approval policy for emergency access (updated)"
        id           = "107ebed2-c54c-4c00-9e3e-9435a04fb179"
        name         = "approval_policy"
        tags         = [
            "approval",
            "emergency",
        ]
        # (8 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # cyral_policy_v2.global_policy_example will be updated in-place
  ~ resource "cyral_policy_v2" "global_policy_example" {
      ~ description  = "Global policy for finance users with row limit for PII data" -> "Global policy for finance users with row limit for PII data (updated)"
        id           = "33aa02d2-2007-4dcf-b2dd-4841c7970a2f"
        name         = "global_policy"
        tags         = [
            "finance",
            "global",
        ]
        # (8 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # cyral_policy_v2.local_policy_example will be updated in-place
  ~ resource "cyral_policy_v2" "local_policy_example" {
      ~ description  = "Local policy to allow gym users to read their own data" -> "Local policy to allow gym users to read their own data (updated)"
        id           = "c11d51eb-48f6-47d2-b39d-fdf694060885"
        name         = "local_policy"
        tags         = [
            "gym",
            "local",
        ]
        # (8 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 3 to change, 0 to destroy.

Changes to Outputs:
  - policy_description = "Local policies for users table access" -> null
  - policy_document    = jsonencode(
        {
          - governedData = {
              - locations = [
                  - "gym_db.users",
                ]
            }
          - readRules    = [
              - {
                  - conditions  = [
                      - {
                          - attribute = "identity.userGroups"
                          - operator  = "contains"
                          - value     = "USERS"
                        },
                    ]
                  - constraints = {
                      - datasetRewrite = "SELECT * FROM ${dataset} WHERE email = '${identity.endUserEmail}'"
                    }
                },
              - {
                  - conditions  = []
                  - constraints = {}
                },
            ]
        }
    ) -> null
  - policy_name        = "policy1" -> null

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

cyral_policy_v2.global_policy_example: Modifying... [id=33aa02d2-2007-4dcf-b2dd-4841c7970a2f]
cyral_policy_v2.approval_policy_example: Modifying... [id=107ebed2-c54c-4c00-9e3e-9435a04fb179]
cyral_policy_v2.local_policy_example: Modifying... [id=c11d51eb-48f6-47d2-b39d-fdf694060885]
cyral_policy_v2.approval_policy_example: Modifications complete after 7s [id=107ebed2-c54c-4c00-9e3e-9435a04fb179]
cyral_policy_v2.local_policy_example: Modifications complete after 7s [id=c11d51eb-48f6-47d2-b39d-fdf694060885]
cyral_policy_v2.global_policy_example: Modifications complete after 7s [id=33aa02d2-2007-4dcf-b2dd-4841c7970a2f]

Apply complete! Resources: 0 added, 3 changed, 0 destroyed.
goran@goran-engdahl-ThinkPad-P1-Gen-6:~/work/terraform$ 

Deleting the policies

goran@goran-engdahl-ThinkPad-P1-Gen-6:~/work/terraform$ terraform apply
cyral_policy_v2.local_policy_example: Refreshing state... [id=c11d51eb-48f6-47d2-b39d-fdf694060885]
cyral_policy_v2.approval_policy_example: Refreshing state... [id=107ebed2-c54c-4c00-9e3e-9435a04fb179]
cyral_policy_v2.global_policy_example: Refreshing state... [id=33aa02d2-2007-4dcf-b2dd-4841c7970a2f]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # cyral_policy_v2.approval_policy_example will be destroyed
  # (because cyral_policy_v2.approval_policy_example is not in configuration)
  - resource "cyral_policy_v2" "approval_policy_example" {
      - created      = {
          - "actor"      = "sa/default/de84bc5b-8204-43cf-a6ab-e101db2c67f4"
          - "actor_type" = "ACTOR_TYPE_API_CLIENT"
          - "timestamp"  = "2024-06-12T12:04:52.411516731Z"
        } -> null
      - description  = "Approval policy for emergency access (updated)" -> null
      - document     = jsonencode(
            {
              - approvals  = {
                  - delete = {
                      - locations = [
                          - "gym_db.users",
                        ]
                      - tags      = [
                          - "PII",
                        ]
                    }
                  - read   = {
                      - locations = [
                          - "gym_db.users",
                        ]
                      - tags      = [
                          - "PII",
                        ]
                    }
                  - update = {
                      - locations = [
                          - "gym_db.users",
                        ]
                      - tags      = [
                          - "PII",
                        ]
                    }
                }
              - conditions = [
                  - {
                      - attribute = "role"
                      - operator  = "equals"
                      - value     = "admin"
                    },
                ]
              - identity   = {
                  - email = "gym_owner@example.com"
                }
            }
        ) -> null
      - enabled      = true -> null
      - enforced     = false -> null
      - id           = "107ebed2-c54c-4c00-9e3e-9435a04fb179" -> null
      - last_updated = {
          - "actor"      = "sa/default/de84bc5b-8204-43cf-a6ab-e101db2c67f4"
          - "actor_type" = "ACTOR_TYPE_API_CLIENT"
          - "timestamp"  = "2024-06-12T12:11:47.505772598Z"
        } -> null
      - name         = "approval_policy" -> null
      - tags         = [
          - "approval",
          - "emergency",
        ] -> null
      - type         = "approval" -> null
      - valid_from   = "2024-06-12T00:00:00Z" -> null
      - valid_until  = "2024-06-13T00:00:00Z" -> null

      - scope {
          - repo_ids = [
              - "gym_repo",
            ] -> null
        }
    }

  # cyral_policy_v2.global_policy_example will be destroyed
  # (because cyral_policy_v2.global_policy_example is not in configuration)
  - resource "cyral_policy_v2" "global_policy_example" {
      - created      = {
          - "actor"      = "sa/default/de84bc5b-8204-43cf-a6ab-e101db2c67f4"
          - "actor_type" = "ACTOR_TYPE_API_CLIENT"
          - "timestamp"  = "2024-06-12T12:04:52.424978774Z"
        } -> null
      - description  = "Global policy for finance users with row limit for PII data (updated)" -> null
      - document     = jsonencode(
            {
              - governedData = {
                  - tags = [
                      - "PII",
                    ]
                }
              - readRules    = [
                  - {
                      - conditions  = [
                          - {
                              - attribute = "identity.userGroups"
                              - operator  = "contains"
                              - value     = "finance"
                            },
                        ]
                      - constraints = {
                          - maxRows = 5
                        }
                    },
                  - {
                      - conditions  = []
                      - constraints = {}
                    },
                ]
            }
        ) -> null
      - enabled      = true -> null
      - enforced     = true -> null
      - id           = "33aa02d2-2007-4dcf-b2dd-4841c7970a2f" -> null
      - last_updated = {
          - "actor"      = "sa/default/de84bc5b-8204-43cf-a6ab-e101db2c67f4"
          - "actor_type" = "ACTOR_TYPE_API_CLIENT"
          - "timestamp"  = "2024-06-12T12:11:47.503363406Z"
        } -> null
      - name         = "global_policy" -> null
      - tags         = [
          - "finance",
          - "global",
        ] -> null
      - type         = "global" -> null
        # (2 unchanged attributes hidden)

      - scope {
          - repo_ids = [
              - "2gaWEAyeKbETyUy1LSx985gVqrk",
            ] -> null
        }
    }

  # cyral_policy_v2.local_policy_example will be destroyed
  # (because cyral_policy_v2.local_policy_example is not in configuration)
  - resource "cyral_policy_v2" "local_policy_example" {
      - created      = {
          - "actor"      = "sa/default/de84bc5b-8204-43cf-a6ab-e101db2c67f4"
          - "actor_type" = "ACTOR_TYPE_API_CLIENT"
          - "timestamp"  = "2024-06-12T12:04:52.418589521Z"
        } -> null
      - description  = "Local policy to allow gym users to read their own data (updated)" -> null
      - document     = jsonencode(
            {
              - governedData = {
                  - locations = [
                      - "gym_db.users",
                    ]
                }
              - readRules    = [
                  - {
                      - conditions  = [
                          - {
                              - attribute = "identity.userGroups"
                              - operator  = "contains"
                              - value     = "users"
                            },
                        ]
                      - constraints = {
                          - datasetRewrite = "SELECT * FROM ${dataset} WHERE email = '${identity.endUserEmail}'"
                        }
                    },
                ]
            }
        ) -> null
      - enabled      = true -> null
      - enforced     = true -> null
      - id           = "c11d51eb-48f6-47d2-b39d-fdf694060885" -> null
      - last_updated = {
          - "actor"      = "sa/default/de84bc5b-8204-43cf-a6ab-e101db2c67f4"
          - "actor_type" = "ACTOR_TYPE_API_CLIENT"
          - "timestamp"  = "2024-06-12T12:11:47.507796234Z"
        } -> null
      - name         = "local_policy" -> null
      - tags         = [
          - "gym",
          - "local",
        ] -> null
      - type         = "local" -> null
        # (2 unchanged attributes hidden)

      - scope {
          - repo_ids = [
              - "2gaWEAyeKbETyUy1LSx985gVqrk",
            ] -> null
        }
    }

Plan: 0 to add, 0 to change, 3 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

cyral_policy_v2.local_policy_example: Destroying... [id=c11d51eb-48f6-47d2-b39d-fdf694060885]
cyral_policy_v2.approval_policy_example: Destroying... [id=107ebed2-c54c-4c00-9e3e-9435a04fb179]
cyral_policy_v2.global_policy_example: Destroying... [id=33aa02d2-2007-4dcf-b2dd-4841c7970a2f]
cyral_policy_v2.local_policy_example: Destruction complete after 1s
cyral_policy_v2.global_policy_example: Destruction complete after 1s
cyral_policy_v2.approval_policy_example: Destruction complete after 2s

Apply complete! Resources: 0 added, 0 changed, 3 destroyed.

Import example

goran@goran-engdahl-ThinkPad-P1-Gen-6:~/work/terraform$ cp import.tf_example import.tf
goran@goran-engdahl-ThinkPad-P1-Gen-6:~/work/terraform$ terraform import cyral_policy_v2.imported_local_policy POLICY_TYPE_LOCAL/a296e3e5-cdb9-496c-a7bd-3088921be32f
cyral_policy_v2.imported_local_policy: Importing from ID "POLICY_TYPE_LOCAL/a296e3e5-cdb9-496c-a7bd-3088921be32f"...
cyral_policy_v2.imported_local_policy: Import prepared!
  Prepared cyral_policy_v2 for import
cyral_policy_v2.imported_local_policy: Refreshing state... [id=a296e3e5-cdb9-496c-a7bd-3088921be32f]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

goran@goran-engdahl-ThinkPad-P1-Gen-6:~/work/terraform$ grep -A10 a296e3e5-cdb9-496c-a7bd-3088921be32f terraform.tfstate
            "id": "a296e3e5-cdb9-496c-a7bd-3088921be32f",
            "last_updated": {
              "actor": "sa/default/de84bc5b-8204-43cf-a6ab-e101db2c67f4",
              "actor_type": "ACTOR_TYPE_API_CLIENT",
              "timestamp": "2024-06-12T09:24:11.790979830Z"
            },
            "name": "local_policy",
            "scope": [
              {
                "repo_ids": [
                  "2gaWEAyeKbETyUy1LSx985gVqrk"

[gengdahlCyral]

Not sure how we want to deprecate, tried update md file - but that is overwritten part of make targets. @wcmjunior @yoursnerdly Other than that I think we are good.

[sonarcloud[bot]]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[wcmjunior]

Changes I've made:

• Deprecated the old resource cyral_poliy
• Fixed a bug introduced in the refactoring
• Fixed docs as the example would not work if deployed as is
• Bumped dependencies and golang