Add Test Evidence for PR https://github.com/prancer-io/prancer-compliance-test/pull/542

[rezoan]

• Link the git url of the respective template in the description.
  • https://github.com/prancer-io/prancer-armof/tree/master/ACI
  • https://github.com/prancer-io/prancer-armof/tree/master/traffic-manager
  • https://github.com/prancer-io/prancer-armof/tree/master/SQL/SQL-Server
  • https://github.com/prancer-io/prancer-terramerra/tree/master/azure/sql_servers
  • https://github.com/prancer-io/prancer-terramerra/tree/master/azure/diagnosticsettings
  • https://github.com/prancer-io/prancer-terramerra/tree/master/azure/container-instance
  • https://github.com/prancer-io/prancer-terramerra/tree/master/azure/traffic-manager
• Add screenshot of passed scenario in the description/comment. (Farshid Mahdavipour suggested: prancer cli text output of passed scenario for running a single testcase)
• Add screenshot of failed scenario in the description/comment. (Farshid Mahdavipour suggested: prancer cli text output of failed scenario for running a single testcase)

[rezoan]

PR-AZR-CLD-ACI-001 failed case:

[rezoan]

Pass case of PR-AZR-CLD-ACI-001

[rezoan]

Failed case of PR-AZR-CLD-ACI-002

[rezoan]

Pass case scenario of PR-AZR-CLD-ACI-002

[rezoan]

failed case scenario of PR-AZR-CLD-ACI-003

[rezoan]

pass case scenario of PR-AZR-CLD-ACI-003

[rezoan]

Failed case of PR-AZR-ARM-ACI-001

[rezoan]

Pass case of PR-AZR-ARM-ACI-001

[rezoan]

Failed case scenario of PR-AZR-ARM-ACI-002

[rezoan]

Pass case scenario PR-AZR-ARM-ACI-002

[rezoan]

Failed case scenario of PR-AZR-ARM-ACI-003

[rezoan]

Passed case scenario of PR-AZR-ARM-ACI-003

[rezoan]

Failed case scenario of PR-AZR-ARM-MNT-013

[rezoan]

Pass case scenario of PR-AZR-ARM-MNT-013

[rezoan]

Failed case scenario of PR-AZR-ARM-SQL-065

[rezoan]

Pass case scenario of PR-AZR-ARM-SQL-065

[rezoan]

Pass case scenario of PR-AZR-CLD-SQL-065

[rezoan]

Failed case scenario of PR-AZR-CLD-SQL-065

[rezoan]

Failed case of PR-AZR-TRF-SQL-071

[rezoan]

Pass case of PR-AZR-TRF-SQL-071

[rezoan]

Failed case of PR-AZR-TRF-SQL-072

[rezoan]

Passed case of PR-AZR-TRF-SQL-072

[rezoan]

Failed case scenario of PR-AZR-TRF-ACI-001

[rezoan]

Passed case scenario of PR-AZR-TRF-ACI-001

[rezoan]

Failed case scenario of PR-AZR-TRF-ACI-002

[rezoan]

Passed case scenario of PR-AZR-TRF-ACI-002

[rezoan]

Failed case scenario of PR-AZR-TRF-ACI-003

[rezoan]

Passed case scenario of PR-AZR-TRF-ACI-003

[rezoan]

Added failed case scenario of PR-AZR-ARM-MNT-013

[rezoan]

Added passed case scenario PR-AZR-ARM-MNT-013

[rezoan]

Terraform prancer cli test output:

2023-01-22 02:21:33,063 - SNAPSHOTS COMPLETE:
2023-01-22 02:21:33,270 -   TESTID: PR-AZR-TRF-ACI-001
2023-01-22 02:21:33,272 -       SNAPSHOTID: TRF_TEMPLATE_SNAPSHOTpaBBR55
2023-01-22 02:21:33,272 -       PATHS: 
2023-01-22 02:21:33,272 -            /azure/container-instance/terraform.tfvars
2023-01-22 02:21:33,272 -            /azure/container-instance/variables.tf
2023-01-22 02:21:33,272 -            /azure/container-instance/provider.tf
2023-01-22 02:21:33,272 -            /azure/container-instance/main.tf
2023-01-22 02:21:33,273 -       TITLE: Ensure Azure Container Instance is configured with virtual network
2023-01-22 02:21:33,273 -       DESCRIPTION: This policy identifies Azure Container Instances (ACI) that are not configured with a virtual network. Making container instances public makes an internet routable network. By deploying container instances into an Azure virtual network, your containers can communicate securely with other resources in the virtual network. So it is recommended to configure all your container instances within a virtual network.<br><br>For more details:<br>https://docs.microsoft.com/en-us/azure/container-instances/container-instances-vnet
2023-01-22 02:21:33,273 -       RULE: file(container_instance.rego)
2023-01-22 02:21:33,273 -       ERROR: Azure Container Instance is currently not configured with virtual network
2023-01-22 02:21:33,273 -       REMEDIATION: In 'azurerm_container_group' resource, set the value to 'private' at property 'ip_address_type' to fix the issue. Visit <a href='https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/container_group#ip_address_type' target='_blank'>here</a> for details.
2023-01-22 02:21:33,274 -       RESULT: failed
2023-01-22 02:21:33,333 -   TESTID: PR-AZR-TRF-ACI-002
2023-01-22 02:21:33,334 -       SNAPSHOTID: TRF_TEMPLATE_SNAPSHOTpaBBR55
2023-01-22 02:21:33,334 -       PATHS: 
2023-01-22 02:21:33,334 -            /azure/container-instance/terraform.tfvars
2023-01-22 02:21:33,334 -            /azure/container-instance/variables.tf
2023-01-22 02:21:33,334 -            /azure/container-instance/provider.tf
2023-01-22 02:21:33,335 -            /azure/container-instance/main.tf
2023-01-22 02:21:33,335 -       TITLE: Ensure Azure Container Instance is configured with managed identity
2023-01-22 02:21:33,335 -       DESCRIPTION: This policy identifies Azure Container Instances (ACI) that are not configured with the managed identity. The managed identity is authenticated with Azure AD, developers don't have to store any credentials in code. So It is recommended to configure managed identity on all your container instances.<br><br>For more details:<br>https://docs.microsoft.com/en-us/azure/container-instances/container-instances-managed-identity
2023-01-22 02:21:33,335 -       RULE: file(container_instance.rego)
2023-01-22 02:21:33,335 -       ERROR: azurerm_container_group property 'identity.type' need to be exist. Its missing from the resource.
2023-01-22 02:21:33,335 -       REMEDIATION: In 'azurerm_container_group' resource, set the value to 'SystemAssigned'/'UserAssigned' at property 'identity.type' to fix the issue. Visit <a href='https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/container_group#identity' target='_blank'>here</a> for details.
2023-01-22 02:21:33,336 -       RESULT: failed
2023-01-22 02:21:33,407 -   TESTID: PR-AZR-TRF-ACI-003
2023-01-22 02:21:33,408 -       SNAPSHOTID: TRF_TEMPLATE_SNAPSHOTpaBBR55
2023-01-22 02:21:33,408 -       PATHS: 
2023-01-22 02:21:33,408 -            /azure/container-instance/terraform.tfvars
2023-01-22 02:21:33,408 -            /azure/container-instance/variables.tf
2023-01-22 02:21:33,408 -            /azure/container-instance/provider.tf
2023-01-22 02:21:33,408 -            /azure/container-instance/main.tf
2023-01-22 02:21:33,409 -       TITLE: Ensure Azure Container Instance usage custom managed key for encryption
2023-01-22 02:21:33,409 -       DESCRIPTION: Secure your containers with greater flexibility using customer-managed keys. When you specify a customer-managed key, that key is used to protect and control access to the key that encrypts your data. Using customer-managed keys provides additional capabilities to control rotation of the key encryption key or cryptographically erase data.
2023-01-22 02:21:33,409 -       RULE: file(container_instance.rego)
2023-01-22 02:21:33,409 -       ERROR: azurerm_container_group property 'key_vault_key_id' need to be exist. Its missing from the resource.
2023-01-22 02:21:33,409 -       REMEDIATION: In 'azurerm_container_group' resource, set the target key vault key id at property 'key_vault_key_id' to fix the issue. Visit <a href='https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/container_group#key_vault_key_id' target='_blank'>here</a> for details.
2023-01-22 02:21:33,409 -       RESULT: failed
2023-01-22 02:21:33,709 -   TESTID: PR-AZR-TRF-MNT-013
2023-01-22 02:21:33,710 -       SNAPSHOTID: TRF_TEMPLATE_SNAPSHOTpaBBR47
2023-01-22 02:21:33,710 -       PATHS: 
2023-01-22 02:21:33,711 -            /azure/vmss/vars.tf
2023-01-22 02:21:33,711 -            /azure/vmss/provider.tf
2023-01-22 02:21:33,711 -            /azure/vmss/main.tf
2023-01-22 02:21:33,711 -       TITLE: Azure Traffic Manager diagnostic logs should be enabled
2023-01-22 02:21:33,712 -       DESCRIPTION: Diagnostic settings for Azure Traffic Manager used to stream resource logs to a Log Analytics workspace. this policy will identify any Azure Traffic Manager which has this diagnostic settings missing or misconfigured.
2023-01-22 02:21:33,712 -       RULE: file(diagnosticsettings.rego)
2023-01-22 02:21:33,712 -       ERROR: azurerm_traffic_manager_profile's azurerm_monitor_diagnostic_setting and its property block 'log' need to be exist. its currently missing from the resource.
2023-01-22 02:21:33,712 -       REMEDIATION: In 'azurerm_monitor_diagnostic_setting' resource, make sure 'log' block exist and 'target_resource_id' contains id of target 'azurerm_traffic_manager_profile' resource to fix the issue. please visit <a href='https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/monitor_diagnostic_setting#log' target='_blank'>here</a> for details.
2023-01-22 02:21:33,713 -       RESULT: failed
2023-01-22 02:21:33,881 -   TESTID: PR-AZR-TRF-MNT-013
2023-01-22 02:21:33,882 -       SNAPSHOTID: TRF_TEMPLATE_SNAPSHOTpaBBR50
2023-01-22 02:21:33,883 -       PATHS: 
2023-01-22 02:21:33,883 -            /azure/traffic-manager/terraform.tfvars
2023-01-22 02:21:33,883 -            /azure/traffic-manager/variables.tf
2023-01-22 02:21:33,883 -            /azure/traffic-manager/provider.tf
2023-01-22 02:21:33,883 -            /azure/traffic-manager/main.tf
2023-01-22 02:21:33,883 -       TITLE: Azure Traffic Manager diagnostic logs should be enabled
2023-01-22 02:21:33,883 -       DESCRIPTION: Diagnostic settings for Azure Traffic Manager used to stream resource logs to a Log Analytics workspace. this policy will identify any Azure Traffic Manager which has this diagnostic settings missing or misconfigured.
2023-01-22 02:21:33,884 -       RULE: file(diagnosticsettings.rego)
2023-01-22 02:21:33,884 -       ERROR: azurerm_traffic_manager_profile's azurerm_monitor_diagnostic_setting and its property block 'log' need to be exist. its currently missing from the resource.
2023-01-22 02:21:33,884 -       REMEDIATION: In 'azurerm_monitor_diagnostic_setting' resource, make sure 'log' block exist and 'target_resource_id' contains id of target 'azurerm_traffic_manager_profile' resource to fix the issue. please visit <a href='https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/monitor_diagnostic_setting#log' target='_blank'>here</a> for details.
2023-01-22 02:21:33,885 -       RESULT: failed
2023-01-22 02:21:34,024 -   TESTID: PR-AZR-TRF-SQL-071
2023-01-22 02:21:34,025 -       SNAPSHOTID: TRF_TEMPLATE_SNAPSHOTpaBBR37
2023-01-22 02:21:34,026 -       PATHS: 
2023-01-22 02:21:34,027 -            /azure/mssql_servers/terraform.tfvars
2023-01-22 02:21:34,027 -            /azure/mssql_servers/vars.tf
2023-01-22 02:21:34,027 -            /azure/mssql_servers/provider.tf
2023-01-22 02:21:34,028 -            /azure/mssql_servers/main.tf
2023-01-22 02:21:34,028 -       TITLE: Ensure that SQL Server configured with a virtual network
2023-01-22 02:21:34,030 -       DESCRIPTION: This policy audits any SQL Server not configured to use a virtual network service endpoint.
2023-01-22 02:21:34,031 -       RULE: file(sql_servers.rego)
2023-01-22 02:21:34,032 -       ERROR: Make sure resource azurerm_mssql_server and azurerm_mssql_virtual_network_rule both exist and linked. either related resource or link is missing.
2023-01-22 02:21:34,033 -       REMEDIATION: In 'azurerm_mssql_virtual_network_rule' resource, make sure property 'server_id' exist and has id of target 'azurerm_mssql_server' resource to fix the issue. please visit <a href='https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/mssql_virtual_network_rule#server_id' target='_blank'>here</a> for details.
2023-01-22 02:21:34,033 -       RESULT: failed
2023-01-22 02:21:34,174 -   TESTID: PR-AZR-TRF-SQL-072
2023-01-22 02:21:34,176 -       SNAPSHOTID: TRF_TEMPLATE_SNAPSHOTpaBBR38
2023-01-22 02:21:34,178 -       PATHS: 
2023-01-22 02:21:34,179 -            /azure/sql_servers/terraform.tfvars
2023-01-22 02:21:34,182 -            /azure/sql_servers/vars.tf
2023-01-22 02:21:34,183 -            /azure/sql_servers/provider.tf
2023-01-22 02:21:34,184 -            /azure/sql_servers/main.tf
2023-01-22 02:21:34,184 -       TITLE: Ensure that SQL Server configured with a virtual network
2023-01-22 02:21:34,185 -       DESCRIPTION: This policy audits any SQL Server not configured to use a virtual network service endpoint.
2023-01-22 02:21:34,186 -       RULE: file(sql_servers.rego)
2023-01-22 02:21:34,187 -       ERROR: Make sure resource azurerm_sql_server and azurerm_sql_virtual_network_rule both exist and linked. either related resource or link is missing.
2023-01-22 02:21:34,187 -       REMEDIATION: In 'azurerm_sql_virtual_network_rule' resource, make sure property 'server_name' exist and has name of target 'azurerm_sql_server' resource to fix the issue. please visit <a href='https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/sql_virtual_network_rule#server_name' target='_blank'>here</a> for details.
2023-01-22 02:21:34,187 -       RESULT: failed
2023-01-22 02:21:34,193 - VALIDATION COMPLETE:
2023-01-22 02:21:34,224 -  Run Stats: {
  "start": "2023-01-22 02:19:46",
  "end": "2023-01-22 02:21:34",
  "remote": false,
  "errors": [],
  "host": "ubuntu",
  "timestamp": "2023-01-22 02:19:46",
  "jsonsource": false,
  "database": 0,
  "container": "scenario-terraform-azure",
  "INCLUDESNAPSHOTS": false,
  "SNAPHSHOTIDS": [],
  "INCLUDETESTS": true,
  "TESTIDS": [
    "PR-AZR-TRF-MNT-013",
    "PR-AZR-TRF-ACI-003",
    "PR-AZR-TRF-ACI-002",
    "PR-AZR-TRF-ACI-001",
    "PR-AZR-TRF-SQL-072",
    "PR-AZR-TRF-SQL-071"
  ],
  "ONLYSNAPSHOTS": false,
  "ONLYSNAPSHOTIDS": [],
  "session_id": "session_1674411586311",
  "run_type": "CRAWL_AND_COMPLIANCE",
  "log": null,
  "duration": "107 seconds"
}

[rezoan]

Cloud Prancer CLI test output (output is being shown based on resource availability on cloud):

2023-01-22 19:17:21,112 - SNAPSHOTS COMPLETE:
2023-01-22 19:17:22,213 -   TESTID: PR-AZR-CLD-SQL-065
2023-01-22 19:17:22,222 -       SNAPSHOTID: AZRSNP_400129
2023-01-22 19:17:22,223 -       PATHS: 
2023-01-22 19:17:22,225 -       TITLE: Ensure that SQL Server configured with a virtual network
2023-01-22 19:17:22,225 -       DESCRIPTION: This policy audits any SQL Server not configured to use a virtual network service endpoint.
2023-01-22 19:17:22,226 -       RULE: file(sql_servers.rego)
2023-01-22 19:17:22,232 -       ERROR: Azure SQL Server currently not configured with vnet
2023-01-22 19:17:22,233 -       REMEDIATION: 01. Sign in to the Azure portal.<br>02. Search for and select SQL servers, and then select your server. Under Security, select Networking.<br>03. Under the Public access tab, ensure Public network access is set to Select networks, otherwise the Virtual networks settings are hidden. Select + Add existing virtual network in the Virtual networks section.<br>04. In the new Create/Update pane, fill in the boxes with the names of your Azure resources.<br>05. See the resulting virtual network rule on the Firewall pane.<br>06. Set Allow Azure services and resources to access this server to No.
2023-01-22 19:17:22,235 -       RESULT: failed
2023-01-22 19:17:22,270 - VALIDATION COMPLETE:
2023-01-22 19:17:22,310 -  Run Stats: {
  "start": "2023-01-22 18:36:15",
  "end": "2023-01-22 19:17:22",
  "remote": false,
  "errors": [],
  "host": "ubuntu",
  "timestamp": "2023-01-22 18:36:15",
  "jsonsource": false,
  "database": 0,
  "container": "scenario-azure",
  "CLEANING_REPOS": [],
  "INCLUDESNAPSHOTS": false,
  "SNAPHSHOTIDS": [],
  "INCLUDETESTS": true,
  "TESTIDS": [
    "PR-AZR-CLD-ACI-003",
    "PR-AZR-CLD-ACI-002",
    "PR-AZR-CLD-ACI-001",
    "PR-AZR-CLD-SQL-065"
  ],
  "ONLYSNAPSHOTS": false,
  "ONLYSNAPSHOTIDS": [],
  "session_id": "session_1674470175517",
  "run_type": "CRAWL_AND_COMPLIANCE",
  "log": null,
  "duration": "2466 seconds"
}

[rezoan]

ARM prancer CLI test output

• One issue being created at https://github.com/prancer-io/cloud-validation-framework/issues/653
• ACI one is in discussion with Ajay and Farshid in a group conversation. (may need to create another issue under cloud validation framework for this)

2023-01-22 02:28:17,424 - SNAPSHOTS COMPLETE:
2023-01-22 02:28:17,719 -   TESTID: PR-AZR-ARM-SQL-065
2023-01-22 02:28:17,720 -       SNAPSHOTID: ARM_TEMPLATE_SNAPSHOTZgnBN2
2023-01-22 02:28:17,720 -       PATHS: 
2023-01-22 02:28:17,721 -            /SQL/sql-encryption-protector-byok/azuredeploy.json
2023-01-22 02:28:17,721 -            /SQL/sql-encryption-protector-byok/azuredeploy.parameters.json
2023-01-22 02:28:17,721 -       TITLE: Ensure that SQL Server configured with a virtual network
2023-01-22 02:28:17,721 -       DESCRIPTION: This policy audits any SQL Server not configured to use a virtual network service endpoint.
2023-01-22 02:28:17,721 -       RULE: file(sql_servers.rego)
2023-01-22 02:28:17,721 -       ERROR: Azure SQL Server currently not configured with vnet
2023-01-22 02:28:17,722 -       REMEDIATION: In Resource of type 'Microsoft.Sql/servers/virtualNetworkRules' make sure properties.virtualNetworkSubnetId exists and has valid subnet id configured.<br>Please visit <a href='https://learn.microsoft.com/en-us/azure/templates/microsoft.sql/servers/virtualnetworkrules?pivots=deployment-language-arm-template' target='_blank'>here</a> for more details.
2023-01-22 02:28:17,722 -       RESULT: failed
2023-01-22 02:28:17,813 -   TESTID: PR-AZR-ARM-SQL-065
2023-01-22 02:28:17,814 -       SNAPSHOTID: ARM_TEMPLATE_SNAPSHOTZgnBN4
2023-01-22 02:28:17,814 -       PATHS: 
2023-01-22 02:28:17,814 -            /SQL/SQL-Server/sql.azuredeploy.json
2023-01-22 02:28:17,814 -            /SQL/SQL-Server/sql.azuredeploy.parameters.json
2023-01-22 02:28:17,815 -       TITLE: Ensure that SQL Server configured with a virtual network
2023-01-22 02:28:17,815 -       DESCRIPTION: This policy audits any SQL Server not configured to use a virtual network service endpoint.
2023-01-22 02:28:17,815 -       RULE: file(sql_servers.rego)
2023-01-22 02:28:17,815 -       ERROR: Azure SQL Server currently not configured with vnet
2023-01-22 02:28:17,815 -       REMEDIATION: In Resource of type 'Microsoft.Sql/servers/virtualNetworkRules' make sure properties.virtualNetworkSubnetId exists and has valid subnet id configured.<br>Please visit <a href='https://learn.microsoft.com/en-us/azure/templates/microsoft.sql/servers/virtualnetworkrules?pivots=deployment-language-arm-template' target='_blank'>here</a> for more details.
2023-01-22 02:28:17,816 -       RESULT: failed
2023-01-22 02:28:17,819 - VALIDATION COMPLETE:
2023-01-22 02:28:17,828 -  Run Stats: {
  "start": "2023-01-22 02:28:06",
  "end": "2023-01-22 02:28:17",
  "remote": false,
  "errors": [],
  "host": "ubuntu",
  "timestamp": "2023-01-22 02:28:06",
  "jsonsource": false,
  "database": 0,
  "container": "scenario-arm-remote",
  "INCLUDESNAPSHOTS": false,
  "SNAPHSHOTIDS": [],
  "INCLUDETESTS": true,
  "TESTIDS": [
    "PR-AZR-ARM-MNT-013",
    "PR-AZR-ARM-ACI-003",
    "PR-AZR-ARM-ACI-002",
    "PR-AZR-ARM-ACI-001",
    "PR-AZR-ARM-SQL-065"
  ],
  "ONLYSNAPSHOTS": false,
  "ONLYSNAPSHOTIDS": [],
  "session_id": "session_1674412086930",
  "run_type": "CRAWL_AND_COMPLIANCE",
  "log": null,
  "duration": "10 seconds"
}