Weird behavior of annotations field in rancher2_app_v2.monitoring

[Jasstkn]

Hi! I was trying to configure monitoring and the huge output appeared in the console because of the changed hash in the annotations field. Is this expected behavior?

  # module.rancher_pre_prod_cluster_init.rancher2_app_v2.monitoring has been changed
  ~ resource "rancher2_app_v2" "monitoring" {
      ~ annotations                 = {
          ~ "objectset.rio.cattle.io/applied"         = "HASH" -> "HASH"
            # (4 unchanged elements hidden)
        }
        id                          = "c-XXXX.cattle-monitoring-system/rancher-monitoring"
        name                        = "rancher-monitoring"
        # (13 unchanged attributes hidden)
    }

[rawmind0]

Hi @Jasstkn , not clear what you mean. Could you please provide more context around what are you trying to do and what is the issue??

[Jasstkn]

@rawmind0 Hi! My case is following - I deploy monitoring via rancher2_app_v2 and for values I provide external yaml file. Due to this changes everytime when something changes in the file (almost every time because of problem with YAML formatting) I see huuuuge output in terraform like this

 ~ resource "rancher2_app_v2" "monitoring" {
      ~ annotations                 = {
          ~ "objectset.rio.cattle.io/applied"         = "H4sIAAAAAAAA/+y9+ZKjyP04+CpEzUbYPaoSSAhJVIQjfqD7RvfRPeFKIDnEkYjkELIdsa+xr7dPskECOqqqe3psf73+xe78MTMFSeYnP/eVqb89OTAAKgjA0+vfnlzgwKfXJx+4igH9Fwe5ZoB809Wfnsk77AElHaCAILDh3fsXnOAAOk/PT4oPQWAid2U6EAfA8Z5e3dC2n59sIEMbp6sg+QiVAMOg7JuonM1VNhFtAGykk/MaYBWGrzMNFXI1VeO1GmRhvdrgFKbC19RaswY5rfH0j+cn4LooIOv9cGZTfXp9MqDtvADPe3r+7jgUu9B/0SPr6fWJjirP1Mh01b8soeLD4Hc/y5GHjXK6UtmHNgQYlqNK+SM+yxFb/akJfxfj/3h+IqMXUIM+dBWIn16//u0JeOYG+thE7tPrU1R5en6yTDfFwnUz/wS4IcEjo3J8pVGpvqi1KnipQbb+AjgGvIC6DKHahFytyaWMgNzAR7YN/afXwA/h85NsI8WapcC2oQ0DApsGbAz/8ds/np+wB5Xf40HT1VA6RjN9HLShZ6MEpiBVmWrlheFemMaqUn2t1F9r1cNTynKfjKq/VJrpqFrllWXSUSrEim96GTxPkg89kC5HhZ7uAxU+PT/hAAQhfnp98qCrpri/vXJRkGL8E3ApA2BKhtClTBcHwLahWqZaBlQsygwwlc1JyQnlh65ruvrrN5eirFCGSmBTLzfiU98jPaXDgPKQiqkXm/r2lBPwLx8h+fb0zf3mbkxsBpQRBB5+pWndDIxQLivIoT0fOTAwYIhfkAd9ECCfTuF4ub2gNOSTbfihQoSNQi5loJgKEEUEHlLfQoap1ikFuZqphz6kBBv6gQNcoEOfAq5KSbfpCEZSXqVCnKIqMCA1y9cuf3Ofnp98CFTCB79Q72B5wQFQrHRHgwyvmHz+9T3IZNhvf/7nNvzlmQKUknIv2S+FNGoUytB3YQAx5QDX1CAO8DP1tecDDbggWyhdJ/ubLPSFUgE2ZAR8FT8THHy9Q4If2hDfALwtnmoAFSn47hFtgwDigC7QS1Qe7UMF+SlD/pXMRX+hFOTIpgtVKjYDg1KREjrQzRQkWT/jdJzSzfNRZKqQggAn6d8ZKiAFXfUlQC/QVe+3rNghDqBP3fE3WeJuQ9/bypc7It/vvyD4T9Pok2dfyikjLCH8lAf+aepTi47QnnQI26swAKaNKSCjMEgR7CEXuint3xMXpByPCUR/naIAvlIrw8SUYgA/oGJApMiBvp1QqXCr1NsnoLxlw58pF8WUD7OBAaIc5ENKsSFIP/ehljImFRggoMygUDCZILy928xbSunUzGQS8UzoZrpUbJiKQX1CD8pMxdtOKOTC237Lf0339csvlORDH57CVJtAnD57ueeTSrlSL6XP+tB2KLaUf9SDAbWAHqIGrobSZ29vbwpyMbLhNze1P5SfvgWqSt0hRUGOE7pmkFAfOev2spwTN/UgUhNPEIjvpw09FQSQLEqIk/LL1+vrG5cQQ4iNTPjSP8i//poOor8QZkgXTUn9IFlX1OQKiWqlELzf5S8EJd/c/4MiK+c0o74uOuOOsOz8dSpMOr99uvv37FlowIftPGiG3/78y8PfXygZ2ijOAL3bfg7E72IgH/dTSGjD1EpCVzEz9hATSoUaCO2UX6/icGVZoKpm+j2wnyk1/zTIBuHXjL2IWGf8laEitZ3wxYGBbyqfC/kPv6ADH0LaAalCo7OVPhn1hSz9KT3uHrpIhS/w7CE/gL+ryW4z3DFqAY3pFrB8b/oMotzAFIbm0zWLMT9cJh/0JUXyClGqiYFswysRUvpRakhUfU4tQuhnCqf844R2YHo2pHK/A//251+KZy/Fs4Lv3rHddYnkdznvNvSnmG/tmn9ABkP3cym8ihbR3z50UAQxlY5LFey9VbxaAwpgjBQTBIXtTUdmrE6gTR1emCnowtN+h5QrML+Lk+vI30dJa9HGuYempr7mnQQCH1IuCvLtkbeFnBJPwUChrVIyTL2dENh2QsxP6luE3ut7xBZ+a7ZPSvFza5j7f5kywuW7iEJBPkQ45dbf/fqPfOchNR/8h77ykQz/4Ae5iP6TnxGH7Q98iaEfmQr8J/YWGMBFZL3vf5azeypAJLJJZf6nBCiLg/4NRmybygzxGSL2mfoJts1s+s+wbT6y/M1tIReTgTZGmcNIVmw/uMnIJct/RwoJEH+VIQ7+6vlACUwFYloJcYBSPwGj0FfgX1WomS6xavjLBzHPUPb7Qp6N+xmtd0814FLwbOIg/f9FpmhS9xFQLowpBxyRT0VZXiD9VsgfZajNX6R/uTqk/mybFqSiSrlaZqmXNL5jIRVVy0yZ+UKZrmoqIFNpILXs0Iep4wjScDdViyAwU2si+xBYKTD5pC6EBM6MOhTIwslysZGujxyqwpbPKdCVWvlMItccrkqtwB6mPnGdKS39mCnXuOxzplyrl8/ljKtUBDHhGxAGyAGBqeS8kTFwHt6mKj5FFOG/NGTKMZMPe6YwohIUUgaICFqLb1JeIt9cmU6GWuqxE97LgvvPFSbwPDuhXrSrh+uDOHdoQ0wUZ5A6338gJKIjsnGGhmfgeDb8bMyL4qv0p7rgr5+p7QQ49v8mIP/XA3tvoP7bQc2s4n8/lDdT/F8O63sr/l8O7oPrkMF69RQKXV3NdTX7TlezP6era1ddzf1ndfV/jarm/vdT1f88yP95Yf33oPe77F/J2b+asf/qGvXFAFOOqfvESQ0QhUMvjeKzzJqRObrEY7VBAP13DhCTz1p5J1SVnxOq6lWo2FSolgHwg2vGNnuaOmq31Av1dp+tb2W89EbExMSU6QY+UkMFqv+fE9A8Vn9JkfZvl9Kcme6oW6QaUhSo0POhkrGPjSUfnRMqMgGlGwgHoetCmzJdSgMR8imkUS4IzAhSq/HyymrpLJ+xiQ5uJYKIKbN8mXnkPz5nP+Yd+zE/xX5s88p+1U/Zj/nIfhJxNf5/jnvguOq/w297r7Fy4vCPpOVTFkrRQDRThrV7ZrzGlG83qi0VH3gwVxadcwB9F9hvBVOGhCu/NzyrQr+VrxWSNDa0YYCpFG+Ywh5UTC0ha2MyltRMshA/+9uCpGwVYkgi5HTkHUdhshr1kAVPAborTWVrm2RJLbTJLNnqJKQlTPIZmwNMAQqH8i27R/IJZDygAuh4qUrPJslyGPhnoHsUwUZGpmZGpnZI+DXl7ev8WujmtVgZknI1hmqKfJJiDG2YStE1pG+Wz+VzQeO8wgTcq/wE6NNC0PuqLaZcqECMgZ+kkyNfhf69bQsM6JSprunj4Pl+7nTRrIB5BQhp6Raz3X2oBf3LaSXq5aVYqFHmysxdQhc+7Lr5H4HhK1nmllaemK7pAJvyYToHdFX4gOfis4ytr5mvXDOkZHirllOP41Gy6xnLNB5YBhTCZbpUnvC/8u4zdUNSyh5ZyfEUmj7EmdrNkj5/ochy74xEnmWpf3c5lXReONANqKz3JyX6xzrLM2GPAt3X9d8eqPDyoiFfgW8PbOfDl7z5ILeX+XJlaqBliDMzo6EiF5LMmO8jn4pN26aQooR+kcXKqtNZGusebIW0F5FkIlJNzYTqB0tguhGwTfWVaKwyhjZUAuS/UoPsORUBO4SvVFQpj1McLPMBf5uAQDHII/zqAO8rDlIt/lv2n799ewKeV74VscomolP99+3p9dvTRxR+e/rHM0Vm7Jw9H+KUpvj162/vV11kyE339mfXtL/845XSTGirxOQ6ThgA2b6VSQskZlgzUrVn+xCoSdbVAl0FhW4Afag+UyCnl2HiAPnJ2zVhqQA3xR9RTgGiVBhA3zFdmJefC55OZ4+Rb6VzBamMZtP5yLZloFhvhRbJxz8M+sAjZCxRVjGFXJhntlv3yrboGvjaIqlb81I0KJCsNyVm9j+v56aE+TxdS/wUmrQ3/JWkbn9RbvO9BAbMim8vmT/xYl7n+1KmVogU0lL/42oHZBtSyMu0etbBQZoPoEpliiIviX5QVthAccZq+J9Jvu9RSDkgyeyYHxaIvZv1LXWnbrroT5j6el8n/O3Pv9z/meWs7+xdvqerAllkPUrU5NZN8oE8aSilIdtGMaENwU0KES42Cs8ewlkF4ON8f7pVlnHBPAR0gsC/UxLwgZMyI/V3qn1rACN/ZdWEv6fDXtJ/qPy/L49/ZSPeUrmcRdD3TRW+UX9PFTnpqwGZx0L0yq0qkVtpHECgPtjjbHScsrULHJLKt1MLkZUUcK6WPpRD/v727ZPes29PbzfgSBfZPYTF/79Xd7eWs79Tb9++23L47ekt37puIxnYZV8GSjn1nBbIhric6eR0oVamnYv6TDqEamWNRGQosRgphQu/K3XJA+gDJcjY/84othb4OWeSCfDyjpvMjcQpvIEfwh/...

[rawmind0]

@Jasstkn , the diff field you are attaching is not significant. It should be any other diff, due to diff on annotations containing cattle.io are supressed, https://github.com/rancher/terraform-provider-rancher2/blob/ec17aecf3d86a3482bb3c33b74ac06dad4232d09/rancher2/schema_common.go#L23

As explained in the issue you are referencing, values yaml diff are checked at low level, nos as string, so just showing "real" diffs. Could you please provide you app v2 tf definition and full diif??

[Jasstkn]

@Jasstkn , the diff field you are attaching is not significant. It should be any other diff, due to diff on annotations containing cattle.io are supressed,

https://github.com/rancher/terraform-provider-rancher2/blob/ec17aecf3d86a3482bb3c33b74ac06dad4232d09/rancher2/schema_common.go#L23

As explained in the issue you are referencing, values yaml diff are checked at low level, nos as string, so just showing "real" diffs. Could you please provide you app v2 tf definition and full diif??

@rawmind0 sure!

resource "rancher2_app_v2" "monitoring" {
  cluster_id = rancher2_cluster.rancher_managed_cluster.id
  name            = "rancher-monitoring"
  namespace       = "cattle-monitoring-system"
  repo_name       = "rancher-charts"
  chart_name      = "rancher-monitoring"
  chart_version   = "14.5.100"
  cleanup_on_fail = true
  values          = file("${path.cwd}/files/rancher-monitoring.yaml")
}

Full log is below log.txt

[rawmind0]

@Jasstkn thanks for the update.. Are you migrating cluster monitoring from v1 to v2??

This seems the only change that the tf plan is calculating. App v2 annotation diff seems suppressed and no any diff at values argument..

...
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place

Terraform will perform the following actions:

  # module.rancher_sandbox_cluster_init.rancher2_cluster.rancher_managed_cluster will be updated in-place
  ~ resource "rancher2_cluster" "rancher_managed_cluster" {
        id                         = "c-XXXX"
        name                       = "sandbox"
        # (15 unchanged attributes hidden)

      - cluster_monitoring_input {
          - answers = {
              - "exporter-kubelets.https"                   = "true"
              - "exporter-node.enabled"                     = "true"
              - "exporter-node.ports.metrics.port"          = "9796"
              - "exporter-node.resources.limits.cpu"        = "50m"
              - "exporter-node.resources.limits.memory"     = "64Mi"
              - "grafana.persistence.enabled"               = "true"
              - "grafana.persistence.size"                  = "10Gi"
              - "grafana.persistence.storageClass"          = "default"
              - "operator.resources.limits.memory"          = "500Mi"
              - "prometheus.persistence.enabled"            = "true"
              - "prometheus.persistence.size"               = "50Gi"
              - "prometheus.persistence.storageClass"       = "default"
              - "prometheus.persistent.useReleaseName"      = "true"
              - "prometheus.resources.core.limits.cpu"      = "1000m"
              - "prometheus.resources.core.limits.memory"   = "1500Mi"
              - "prometheus.resources.core.requests.cpu"    = "750m"
              - "prometheus.resources.core.requests.memory" = "750Mi"
              - "prometheus.retention"                      = "7d"
            } -> null
        }

        # (3 unchanged blocks hidden)
    }

cluster_monitoring_input just makes effect for cluster monitoring v1. To customize v2 monitoring you should do it in that way, https://registry.terraform.io/providers/rancher/rancher2/latest/docs/guides/apps_marketplace#rancher-monitoring

[Jasstkn]

@rawmind0 Thank you for your answer. No, I'm not migrating. Monitoring was initially deployed like that. :( I don't understand what can trigger such output. What steps can you suggest to mitigate the issue?

[rawmind0]

No, I'm not migrating. Monitoring was initially deployed like that. :( I don't understand what can trigger such output. What steps can you suggest to mitigate the issue?

You mean that the cluster was initially deployed with monitoring v1 enabled?? Once the plan is generated are you applying it??

[Jasstkn]

No, I'm not migrating. Monitoring was initially deployed like that. :( I don't understand what can trigger such output. What steps can you suggest to mitigate the issue?

You mean that the cluster was initially deployed with monitoring v1 enabled?? Once the plan is generated are you applying it??

Nope, I initially deployed monitoring v2 via cluster explorer view. Yes, I applied plan already few times with such output but it's still worrying me.

[Jasstkn]

@rawmind0 I've tried to apply the plan, it has been applied successfully. But when I run terraform plan right after apply, it still shows me this huge output :(