splunk / terraform-provider-victorops

Manage VictorOps teams, users, escalation policies and routing keys using Terraform
Mozilla Public License 2.0
12 stars 11 forks source link

Support Rotation policy resource for seamless terraform workflow #12

Open rdsubhas opened 3 years ago

rdsubhas commented 3 years ago

Describe the problem

This provider exists to make managing Splunk easier, which is great! We are using this to create teams in victorops. Although the docs say:


// Create a new team
resource "victorops_team" "team_vikings" { ...

// Create escalation policies for existing VO rotation (created using portal)
// Note: You need to fetch an existing Rotation Group Slug for using the Escalation Policy
resource "victorops_escalation_policy" "high_severity" {
    ...
    entries = [
      {
        type = "rotationGroup"
        slug = "rtg-wvvhXshpvaRdn7jM"
      }
    ]
  } 
}

// Create routing keys to push alerts to our escalation policies
resource "victorops_routing_key" "viking_high_severity" { ...

This theoretically looks fine. But in reality, one cannot create rotation group without creating the team first.

So the actual workflow becomes like this:

If we're trying to manage 10 teams in VictorOps or keeping them in-sync with a team directory, the workflow becomes a constant back and forth of broken jobs and partial commits, and having to coordinate across multiple teams whether or not they have finished creating their groups and so on.

What would be good is just to have the basic flow working without breaks in the middle, i.e. have a routing key and escalation policy that we can start using everywhere, and let the teams to configure their shifts in the rotation at their own pace.

Given

  1. Given that the objective of this terraform provider is to have a good automation workflow, unfortunately this middle-of-the-road human blocker for rotation groups is making the automation extremely fragile. Instead of messing around with 2 or 3 commits, it's just more straight forward to do all the clicks in VictorOps itself, undermining the usefulness of this effort.

  2. "Creating a Rotation Policy" is a harmless operation. It does not hurt on-call. Rotation groups are always created with empty members. VictorOps tells this clearly.

image

During creation, the Rotation policy is just pure metadata (shift type, etc). It does not need any user for creation. Users can be selected later after creation.

Request

TL;DR:

Given the objective of this plugin is to make automation smooth for Terraform, having such a cumbersome, manual failure point on creating Rotation groups makes the workflow very flaky, with hard rollbacks, multi-commits, copying bits and pieces here and there, constantly having to intervene. In practical real life, it's easier to just do everything in the UI, instead of having to go back and forth.

Being able to create an empty Rotation Policy with no members will make the terraform workflow very smooth and unblock the entire terraform experience.

Thank you for your patience to read this through.