disaster37 / terraform-provider-kibana

Custom provider to handle Kibana API
MIT License
11 stars 21 forks source link

kibana provider get stuck when an existing object (i.e. index-pattern) is manually deleted from infrastructure #8

Closed diaconud007 closed 2 years ago

diaconud007 commented 3 years ago

Hello,

I use kibana provider in order to maintain as code the configuration of my elk clusters.

What I use: Kibana & Elasticsearch : Version 7.9.3 Terraform: v0.14.2

My code: `resource kibana_object "k8s-tools-index-patterns" { for_each = { for index-pattern in var.k8s-tools-index-patterns : index-pattern.name => index-pattern }

name = each.value.name data = templatefile("${path.cwd}/index-patterns/k8s-tools-index-pattern-template.json", { index-pattern = each.value.pattern, index-pattern-name = each.value.name, timeFieldName = each.value.time_field }) deep_reference = each.value.deep_reference export_objects { id = each.value.name type = each.value.type } provider = kibana.platformtools }`

variable "k8s-tools-index-patterns" { type = list(object({ name = string pattern = string deep_reference = string type = string space = string time_field = string })) }

ndjson file: {"type":"index-pattern","id":"${index-pattern-name}","attributes":{"title":"${index-pattern}","timeFieldName": "${timeFieldName}"},"migrationVersion":{"index-pattern":"7.6.0"},"references":[],"updated_at":"2021-02-22T23:13:19.962Z","version":"WzY3OSwxXQ=="}

The provider creates correctly the index patterns from scratch, but there are the following issues:

  1. If I manually delete an index-pattern from kibana, even if I don't make changes into the tf files, at the next tf plan ar tf apply, the provider get stuck and some time gives me 400 BadRequest. As a work arround, I found the following: delete all the index patterns from kibana, delete the object from tf state ( terraform state rm 'kibana_object.k8s-tools-index-patterns') and after date recreate the index-patterns via terraform.

  2. If I have created a list of index patterns with terraform, and after that I want to remove one index-pattern from the list via terraform, the provider delete the object from tf state, but don't delete the object from kibana

Could you please take a look at these issues? Thank you

diaconud007 commented 3 years ago

Nothing here?!?!?

disaster37 commented 3 years ago

Hi, I will check that

disaster37 commented 3 years ago

Hi,

I have tested the following story:

  1. Create index-pattern with terraform:
    
    terraform {
    required_providers {
    kibana = {
      source = "disaster37/kibana"
    }
    }
    }

provider "kibana" { url = "http://kibana:5601" username = "elastic" password = "changeme" }

resource "kibana_object" "test" { name = "terraform-test" data = "{\"id\": \"test\", \"type\": \"index-pattern\",\"attributes\": {\"title\": \"test\"}}" deep_reference = "true" export_types = ["index-pattern"] }


2. Remove manually the index-pattern

curl -X DELETE \ http://kibana:5601/api/saved_objects/index-pattern/test \ -H 'Content-Type: application/json' \ -H 'kbn-xsrf: true' \ -u elastic:changeme



3. Lauch againt the `terraform apply` -> it recrate the resource as expected

So, maybee the last release fix the issue after upgraded terraform SDK to v2 (release 7.12.7) or the story is not the right.

Can you test the same think on your environment ?