Extract Board Query blocks into a query_board resource

Is your feature request related to a problem? Please describe. Working on rewriting a bunch of boards using the terraform provider.
So far so good but I find annoying for the query block in the honecombio_board resource to have to be defined in the board resource itself rather than being referenced from somewhere else.

I'm creating one module for one board:

main.tf -> board definition
[name].tf -> [name] query definition

If there are special visualisations I need to make for one query widget (e.g. stacked graph), I need to edit main.tf I think this mixes things up and add confusion.
On top of it, is a different journey than the UI one cause in the UI you would edit in the query page, not the board one.

Describe the solution you'd like I'd like to maintain separation.
My suggestion is create a honeycombio_query_visualisation resource which ID can be referenced in the honeycombio_board resource much like honeycombio_query_annotation:

resource "honeycombio_query_visualisation" "latency_by_userid" {
    query_id  = honeycombio_query.latency_by_userid.id

    graph_settings {
      utc_xaxis = true
    }
}

resource "honeycombio_board" "overview" {
  name = "Service Overview"

  query {
    query_id  = honeycombio_query.latency_by_userid.id
    query_annotation_id = honeycombio_query_annotation.latency_by_userid.id
    query_visualisation_id = honeycombio_query_visualisation.latency_by_userid.id
  }
}

Ideally I'd be able to do something like this in a main.tf file:

locals {
  queries = toset(["query1", "query2", ...])
}
resource "honeycombio_board" "board" {
  name = "The Board"

  dynamic "query" {
    for_each = local.queries
    content {
      query_id = query.value.query_id
      query_annotation_id = query.value.annotation_id
      query_visualisation_id = query.value.query_visualisation_id
    }
  }
}

and have a number of files each containing the detail of a query including the visualisation options.

Describe alternatives you've considered Dynamic magic which would only make things more confusing IMO.

Additional context

Thanks for this request, @erikologic.

What you are suggesting is step away from the underlying APIs design and HashiCorp recommends not trying to create provider specific abstractions as they just end up becoming painful to debug and maintain.

The best suggestion I have for you as a workaround would be to build an input variable for your module that is map(object(...)). Something like this:

variable "queries" {
  type = map(object({
    name           = string
    description    = string
    dataset        = string
    query          = string
    caption        = string
    graph_settings = set(map(string))
  }))
}

Your module itself would have code looking something like this:

resource "honeycombio_query" "this" {
  for_each = var.queries

  dataset    = each.value.dataset
  query_json = each.value.query
}

resource "honeycombio_query_annotation" "this" {
  for_each = var.queries

  dataset     = each.value.dataset
  query_id    = honeycombio_query.this[each.key].id
  name        = each.value.name
  description = each.value.description
}

resource "honeycombio_board" "this" {
  name = "myboard"

  dynamic "query" {
    for_each = var.queries
    content {
      query_id            = honeycombio_query.this[query.key].id
      query_annotation_id = honeycombio_query_annotation.this[query.key].id
      caption             = query.value.caption

      dynamic "graph_settings" {
        for_each = query.value.graph_settings
        content {
          utc_xaxis           = lookup(graph_settings.value, "utc_xaxis", false)
          stacked_graphs      = lookup(graph_settings.value, "stacked_graphs", false)
          hide_markers        = lookup(graph_settings.value, "hide_markers", false)
          log_scale           = lookup(graph_settings.value, "log_scale", false)
          omit_missing_values = lookup(graph_settings.value, "omit_missing_values", false)
          overlaid_charts     = lookup(graph_settings.value, "overlaid_charts", false)
        }
      }
    }
  }
}

Creating an instance of the module would then end up looking something like this:

module "board" {
  source = "./modules/myboard"

  queries = {
    query1 = {
      name        = "My Query"
      description = "A very nice query looking at things"
      dataset     = var.dataset
      caption     = "This is a caption"
      graph_settings = [{
        utc_xaxis = true
      }]
      query = <<EOF
{
    "time_range": 7200,
    "granularity": 0,
    "breakdowns": [],
    "calculations": [
        {
            "op": "COUNT"
        }
    ],
    "orders": [],
    "havings": [],
    "limit": 1000
}
EOF
    },
    query2 = {
      name        = "My Other Query"
      description = "Query 2"
      dataset     = var.dataset
      caption     = "This is another caption"
      graph_settings = [{
        utc_xaxis      = true
        stacked_graphs = true
      }]
      query = <<EOF
{
    "time_range": 7200,
    "granularity": 0,
    "breakdowns": [
        "app.tenant"
    ],
    "calculations": [
        {
            "op": "COUNT"
        }
    ],
    "filters": [
        {
            "column": "app.error",
            "op": "exists"
        }
    ],
    "filter_combination": "AND",
    "orders": [
        {
            "op": "COUNT",
            "order": "descending"
        }
    ],
    "havings": [],
    "limit": 1000
}
EOF
    },
  }
}

I'll leave this request open however to track it as a future improvement to the Boards API and the provider.

Hey @jharley amazing thanks for the detailed answer.
I understand your motivations and makes sense.

One feedback I received from my team is that is confusing that the API has visualisations defined at board level, but the UI is defined at query level.
e.g. In the UI I would update the stacked_graphs option by clicking on from the board into the query, and updating the query. In the API, I create a query first, then set up the board/query object with the stacked_graphs option in there.

NABD - the world will still spin tomorrow if that doesn't get changed :)

oh sorry I just noticed I already mentioned this previously 🤦

honeycombio / terraform-provider-honeycombio

Extract Board Query blocks into a query_board resource #482