search

cds-snc / notification-terraform

Terraform for notification.canada.ca
MIT License
13 stars 6 forks source link

Terraform New Relic Slack Workflow for Staging Notifications #1430

Closed P0NDER0SA closed 1 month ago

P0NDER0SA commented 1 month ago

Summary | Résumé

Adding a codified TF solution for our Slack Staging notifications

Related Issues | Cartes liées

https://app.zenhub.com/workspaces/notify-planning-core-6411dfb7c95fb80014e0cab0/issues/gh/cds-snc/notification-planning-core/381

Test instructions | Instructions pour tester la modification

  1. ensure terragrunt plan works
  2. navigate to this workflow and get it to send a test notification to slack after terragrunt apply

Release Instructions | Instructions pour le déploiement

None.

Reviewer checklist | Liste de vérification du réviseur

⚠ If boxes cannot be checked off before merging the PR, they should be moved to the "Release Instructions" section with appropriate steps required to verify before release. For example, changes to celery code may require tests on staging to verify that performance has not been affected.

github-actions[bot] commented 1 month ago

Staging: newrelic

✅   Terraform Init: success ✅   Terraform Validate: success ✅   Terraform Format: success ✅   Terraform Plan: success

Plan: 5 to add, 0 to change, 0 to destroy
Show summary | CHANGE | NAME | |--------|--------------------------------------------------------------------------------------| | add | `newrelic_alert_policy.terraform_notify_policy` | | | `newrelic_notification_channel.terraform_notify_channel` | | | `newrelic_notification_destination.terraform_notify_destination` | | | `newrelic_nrql_alert_condition.tf_lambda_api_errors_count_anomaly_unexpected_errors` | | | `newrelic_workflow.terraform_notify_workflow` |
Show plan ```terraform Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # newrelic_alert_policy.terraform_notify_policy will be created + resource "newrelic_alert_policy" "terraform_notify_policy" { + account_id = (known after apply) + id = (known after apply) + incident_preference = "PER_POLICY" + name = "TF Notify Policy - staging" } # newrelic_notification_channel.terraform_notify_channel will be created + resource "newrelic_notification_channel" "terraform_notify_channel" { + account_id = (known after apply) + active = true + destination_id = (known after apply) + id = (known after apply) + name = "Terraform Notify Slack Channel - staging" + product = "IINT" + status = (known after apply) + type = "SLACK_LEGACY" + property { + key = "payload" + label = "Payload Template" + value = jsonencode({}) } } # newrelic_notification_destination.terraform_notify_destination will be created + resource "newrelic_notification_destination" "terraform_notify_destination" { + account_id = 2691974 + active = true + guid = (known after apply) + id = (known after apply) + last_sent = (known after apply) + name = "Terraform Notify Slack Destination - staging" + status = (known after apply) + type = "SLACK_LEGACY" + property { + display_value = "notification-staging-ops" + key = "url" + value = "https://sre-bot.cdssandbox.xyz/hook/d446a982-dd94-4422-a2d1-0d9b6855e853" } } # newrelic_nrql_alert_condition.tf_lambda_api_errors_count_anomaly_unexpected_errors will be created + resource "newrelic_nrql_alert_condition" "tf_lambda_api_errors_count_anomaly_unexpected_errors" { + account_id = 2691974 + aggregation_delay = "300" + aggregation_method = "event_flow" + aggregation_window = 60 + baseline_direction = "upper_and_lower" + enabled = true + entity_guid = (known after apply) + id = (known after apply) + name = "staging - Terraform - [Lambda API] Errors count anomaly (Unexpected Errors)" + policy_id = (known after apply) + type = "baseline" + violation_time_limit = (known after apply) + violation_time_limit_seconds = 86400 + critical { + operator = "above" + threshold = 6 + threshold_duration = 300 + threshold_occurrences = "all" } + nrql { + data_account_id = (known after apply) + query = "SELECT count(*) FROM AwsLambdaInvocationError WHERE (`entityGuid`='MjY5MTk3NHxJTkZSQXxOQXwtNzgwNDUyNTc5NzAyODI1NTcyNw') and error.class NOT IN ('app.v2.errors:BadRequestError','jsonschema.exceptions:ValidationError', 'sqlalchemy.exc:NoResultFound', 'app.authentication.auth:AuthError', 'werkzeug.exceptions:MethodNotAllowed') and error.message NOT LIKE '{\\'result\\': \\'error\\', \\'message\\': {\\'password\\': [\\'Incorrect password\\']}}'" } + warning { + operator = "above" + threshold = 3 + threshold_duration = 300 + threshold_occurrences = "all" } } # newrelic_workflow.terraform_notify_workflow will be created + resource "newrelic_workflow" "terraform_notify_workflow" { + account_id = 2691974 + destinations_enabled = true + enabled = true + enrichments_enabled = true + guid = (known after apply) + id = (known after apply) + last_run = (known after apply) + muting_rules_handling = "DONT_NOTIFY_FULLY_MUTED_ISSUES" + name = "Terraform Notify Workflow - staging" + workflow_id = (known after apply) + destination { + channel_id = (known after apply) + name = (known after apply) + notification_triggers = (known after apply) + type = (known after apply) } + issues_filter { + filter_id = (known after apply) + name = "workflow-filter" + type = "FILTER" + predicate { + attribute = "labels.policyIds" + operator = "EXACTLY_MATCHES" + values = (known after apply) } + predicate { + attribute = "priority" + operator = "EQUAL" + values = [ + "CRITICAL", ] } } } Plan: 5 to add, 0 to change, 0 to destroy. ───────────────────────────────────────────────────────────────────────────── Saved the plan to: plan.tfplan To perform exactly these actions, run the following command to apply: terraform apply "plan.tfplan" ```
github-actions[bot] commented 1 month ago

Staging: quicksight

✅   Terraform Init: success ✅   Terraform Validate: success ✅   Terraform Format: success ✅   Terraform Plan: success ✅   Conftest: success 

Plan: 0 to add, 1 to change, 0 to destroy
Show summary | CHANGE | NAME | |--------|-------------------------------| | update | `aws_s3_object.manifest_file` |
Show plan ```terraform Resource actions are indicated with the following symbols: ~ update in-place Terraform will perform the following actions: # aws_s3_object.manifest_file will be updated in-place ~ resource "aws_s3_object" "manifest_file" { ~ etag = "4f558e8d8cdbbf914a95755cbda61968" -> "221f592f333f2fc284626cfdb8c4bc80" id = "quicksight/s3-manifest-sms-usage.json" tags = {} + version_id = (known after apply) # (11 unchanged attributes hidden) } Plan: 0 to add, 1 to change, 0 to destroy. ───────────────────────────────────────────────────────────────────────────── Saved the plan to: plan.tfplan To perform exactly these actions, run the following command to apply: terraform apply "plan.tfplan" ```
Show Conftest results ```sh WARN - plan.json - main - Missing Common Tags: ["aws_cloudformation_stack.sms-usage-notifications"] WARN - plan.json - main - Missing Common Tags: ["aws_iam_policy.quicksight-rds"] WARN - plan.json - main - Missing Common Tags: ["aws_iam_policy.quicksight-s3-usage"] WARN - plan.json - main - Missing Common Tags: ["aws_iam_policy.quicksight_vpc_connection_ec2"] WARN - plan.json - main - Missing Common Tags: ["aws_iam_policy.quicksight_vpc_connection_iam"] WARN - plan.json - main - Missing Common Tags: ["aws_iam_role.quicksight"] WARN - plan.json - main - Missing Common Tags: ["aws_iam_role.vpc_connection_role"] WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.jobs"] WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.login_events"] WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.notifications"] WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.organisation"] WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.services"] WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.sms_usage"] WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.templates"] WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.users"] WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_source.rds"] WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_source.s3_sms_usage"] WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_vpc_connection.rds"] WARN - plan.json - main - Missing Common Tags: ["aws_s3_object.manifest_file"] 38 tests, 19 passed, 19 warnings, 0 failures, 0 exceptions ```