feedback/bug: time_series panel

[eraac]

I'm not sure is the good place to send a feedback, but I can rewrite this somewhere if needed

First of all, I was very excited by this provider when the announce come, and after few hours to try to reproduce some mimic of our custom "dashboard-builder" I'm still excited by the capabilities of this.

I have try to reproduce a small part of our custom code to do the same thing, a very simple dashboard with 3 panels (1 text, 2 timeseries), and after few hacks/workaround I was able to reproduce the same dashboard and I'm quitte impressed by this.

Here some of difficulty encounters

• stackdriver / cloudwatch queries are not available -> not a big problem, as the JSON is easy to create via a custom object and jsonencode
• the "row panel" is also missing -> same, probably easy to inject inside schemas_core_dashboard via a custom object + jsonencode
• it's seems there is a constant drift with the grafana_dashboard resource, because of the uid
• for the time_series panel
  • options.legend.(display_name|position) are required to set when options.legend.show_legend is false
  • field_config.defaults.max can't be null if field_config.defaults.min is 0, the provider force 0 as default value, so we can reproduce the "auto" value from the UI when unset
  • field_config.defaults.custom.axis_grid_show it's a boolean, but there are 3 options in UI
  • true: show grid (for left and right axis)
  • false: hide grid
  • auto (unset): show only left grid <- in Terraform we can't unset this value as soon as any value inside field_config.defaults.custom is set, because of default value
  • default values are not the same as the UI (maybe it's on purpose)
  • also default values from the provider can "break" the panel, like axisWidth set to 0 (hidden the Y-axis)
  • field_config.overrides stay empty despite value set, in the code it's seems nested attributes are not defined (matcher.id, matcher.options, properties[].id and properties[].value)
  • same issue with transformations

terraform code

**disclaimer**: my workaround are dirty ```hcl data "schemas_core_dashboard" "my_dashboard" { title = "My Dashboard" time = { from = "now-1h" } # the replace with the regex remove keys from the JSON with value '-1' or '"-1"', to unset key and lets Grafana use default value panels = [ data.schemas_panel_text.title.rendered_json, replace( jsonencode( merge( # fieldConfig.override can't be set via the `data.schemas_panel_time_series`, so we inject the object manually, and deal with merge function who erase others key of an overridden map jsondecode(data.schemas_panel_time_series.publish_rate.rendered_json), { fieldConfig = { defaults = jsondecode(data.schemas_panel_time_series.publish_rate.rendered_json)["fieldConfig"]["defaults"], overrides = [{ matcher = { id = "byName", options = "message rate" }, properties = [{ id = "custom.axisPlacement", value = "right" }, { id = "unit", value = "ops" }] }] } } ) ), "/\"[a-zA-Z_]+\":(| )\"?-1\"?,?/", "" ), replace(data.schemas_panel_time_series.average_message_size.rendered_json, "/\"[a-zA-Z_]+\":(| )\"?-1\"?,?/", ""), ] } data "schemas_panel_text" "title" { transparent = true options = { content = "

\n\tmy.topic\n

" } grid_pos = { h = 2 w = 24 } } data "schemas_panel_time_series" "publish_rate" { title = "Publish rate" description = "left: publish request per second
\nright: message publish per second" datasource = { type = "stackdriver" uid = local.datasource_gcp } field_config = { defaults = { custom = { show_points = "never", draw_style = "line", line_interpolation = "linear", line_width = 1, gradient_mode = "none", span_nulls = false, point_size = 5, axis_placement = "auto", axis_color_mode = "text", axis_grid_show = true, axis_width = -1, axis_soft_min = -1 axis_soft_max = -1, } unit = "reqps", color = { mode = "palette-classic" } decimals = -1, min = 0, max = -1, no_value = "-1", } # doesn't work overrides = [ { matcher = { id = "byName", options = "message rate" } properties = [ { id = "custom.axisPlacement", value = "right" }, { id = "unit", value = "ops" } ] } ] } options = { tooltip = { mode = "multi", sort = "none", } legend = { show_legend = true, display_mode = "table", placement = "bottom", calcs = ["mean"], } } targets = [ "{\"aliasBy\":\"publish rate\",\"datasource\":{\"type\":\"stackdriver\",\"uid\":\"xxxx\"},\"queryType\":\"timeSeriesList\",\"refId\":\"A\",\"timeSeriesList\":{\"alignmentPeriod\":\"cloud-monitoring-auto\",\"crossSeriesReducer\":\"REDUCE_NONE\",\"filters\":[\"resource.label.topic_id\",\"=\",\"my.topic\",\"AND\",\"metric.type\",\"=\",\"pubsub.googleapis.com/topic/send_request_count\"],\"groupBys\":[],\"perSeriesAligner\":\"ALIGN_MEAN\",\"preprocessor\":\"rate\",\"projectName\":\"my-project\"}}", "{\"aliasBy\":\"message rate\",\"datasource\":{\"type\":\"stackdriver\",\"uid\":\"xxxxx\"},\"hide\":false,\"queryType\":\"timeSeriesList\",\"refId\":\"B\",\"timeSeriesList\":{\"alignmentPeriod\":\"cloud-monitoring-auto\",\"crossSeriesReducer\":\"REDUCE_NONE\",\"filters\":[\"resource.label.topic_id\",\"=\",\"my.topic\",\"AND\",\"metric.type\",\"=\",\"pubsub.googleapis.com/topic/send_message_operation_count\"],\"groupBys\":[],\"perSeriesAligner\":\"ALIGN_NONE\",\"preprocessor\":\"rate\",\"projectName\":\"my-project\"}}" ] grid_pos = { h = 9 w = 12 x = 0 } } data "schemas_panel_time_series" "average_message_size" { title = "Average message size" description = "Average message size in this topic grouped by minute" datasource = { type = "stackdriver" uid = local.datasource_gcp } field_config = { defaults = { custom = { show_points = "never", draw_style = "line", line_interpolation = "linear", line_width = 1, gradient_mode = "none", span_nulls = false, point_size = 5, axis_placement = "auto", axis_color_mode = "text", axis_grid_show = true, axis_width = -1, axis_soft_min = -1 axis_soft_max = -1, } min = 0, max = -1, no_value = "-1", } } options = { tooltip = { mode = "multi", sort = "none", } legend = { show_legend = false, display_mode = "table", placement = "bottom", } } targets = ["{\"aliasBy\":\"size\",\"datasource\":{\"type\":\"stackdriver\",\"uid\":\"xxxxxx\"},\"queryType\":\"timeSeriesList\",\"refId\":\"A\",\"timeSeriesList\":{\"alignmentPeriod\":\"cloud-monitoring-auto\",\"crossSeriesReducer\":\"REDUCE_MEAN\",\"filters\":[\"resource.label.topic_id\",\"=\",\"my.topic\",\"AND\",\"metric.type\",\"=\",\"pubsub.googleapis.com/topic/message_sizes\"],\"groupBys\":[],\"perSeriesAligner\":\"ALIGN_SUM\",\"preprocessor\":\"none\",\"projectName\":\"my-project\"}}"] grid_pos = { h = 9 w = 12 x = 12 } } resource "grafana_dashboard" "example" { folder = local.folder_uid config_json = data.schemas_core_dashboard.my_dashboard.rendered_json } ```

[AgnesToulet]

Hi! Thanks for this great feedback, this is the perfect place to post it.

• For the missing panels, I created the following issues to keep track of them: https://github.com/grafana/terraform-provider-schemas/issues/45 and https://github.com/grafana/terraform-provider-schemas/issues/46.

it's seems there is a constant drift with the grafana_dashboard resource, because of the uid

• I don't understand this point, could you please elaborate?

• For the time_series panel:
  • options.legend.(display_name|position) are not defined yet in the schema.
  • Similar issue for field_config.defaults.custom.axis_grid_show that seems not correctly defined.
  • "field_config.defaults.max can't be null if field_config.defaults.min is 0" -> I investigated this and found out the reason why it works this way. I created an issue for it: https://github.com/grafana/terraform-provider-schemas/issues/47
  • "default values are not the same as the UI" -> I think most of that if not all is due to the bug above (maybe some discrepancies lie in the schemas as well that are still in an experimental state).
  • transformations / field_config.overrides issues -> I created an issue to fix the generation of these fields: https://github.com/grafana/terraform-provider-schemas/issues/48 though it won't entirely fix them as some of their fields don't have a type in the schemas.

Thanks again for this detailed and clear feedback!

[eraac]

I don't understand this point, could you please elaborate?

Yes, indeed it was incomplete. When you combine schemas_core_dashboard and grafana_dashboard, if the attribute uid isn't set explicitly in the first one, terraform plan|apply will show constant diff between the current state and the terraform state

data "schemas_core_dashboard" "main" {
  # value not set
  #uid   = "my-uid"

  ...
}

resource "grafana_dashboard" "main" {
  ... 

  # here uid isn't set, but set in Grafana (auto generate), so the provider consider there is a difference
  config_json = data.schemas_core_dashboard.main.rendered_json
}

If you apply with the diff, nothing change (Grafana seems smart enough to re-use the current uid)