[Bug] Add support for NDJSON files for opensearch_dashboard_object creation

n6809 commented 1 month ago

The problem

I want to import saved objects to dashboard with terraform resource "opensearch_dashboard_object". For that I have an ndjson file that I reference in the resource section but I think this creates problem maybe ndjson is not yet supported. I have multi tenancy and need to import specific file to each tenant. Here is an example of what I am using:

locals {
  # Read the NDJSON files
 Tenant_1  = file("${path.module}/opensearch-dashboards/Tenants/Tenant_1/Tenant_1_visualization.ndjson")
}

resource "opensearch_dashboard_object" "Tenant_1_visualization" {
   tenant_name = "Tenant_1"
   body        = local.Tenant_1
}

For that I received this error:

Error: "body" contains an invalid JSON: invalid character '{' after top-level value
│ 
│   with module.environment.module.opensearch.opensearch_dashboard_object.Tenant_1_visualization,
│   on ../../modules/aws/opensearch/main.tf line 199, in resource "opensearch_dashboard_object" "Tenant_1_visualization":
│  199:   body        = local.Tenant_1

Or this:

Error: "body" must be an array of objects: json: cannot unmarshal object into Go value of type []interface {}
│ 
│   with module.environment.module.opensearch.opensearch_dashboard_object.Tenant_1_visualization,
│   on ../../modules/aws/opensearch/main.tf line [199] in resource "opensearch_dashboard_object" "Tenant_1_visualization":
│  199:   body        = local.Tenant_1

Here is my dummy ndjson file, I mad esure it's correctly formatted but I read that NDJSON is a bunch of JSON file together in one single file:

{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[],\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.index\"}"},"title":"filter.perf.request_category","uiStateJSON":"{}","version":1,"visState":"{\"title\":\"filter.perf.request_category\",\"type\":\"pie\",\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"params\":{},\"schema\":\"metric\"},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"params\":{\"field\":\"request_category\",\"orderBy\":\"1\",\"order\":\"desc\",\"size\":5,\"otherBucket\":false,\"otherBucketLabel\":\"Other\",\"missingBucket\":false,\"missingBucketLabel\":\"Missing\"},\"schema\":\"segment\"}],\"params\":{\"type\":\"pie\",\"addTooltip\":true,\"addLegend\":true,\"legendPosition\":\"right\",\"isDonut\":true,\"labels\":{\"show\":false,\"values\":true,\"last_level\":true,\"truncate\":100}}}"},"id":"00d32850-0630-11ef-bdbd-2bd8e8deefb6","migrationVersion":{"visualization":"7.10.0"},"references":[{"id":"index-pattern-log.perf","name":"kibanaSavedObjectMeta.searchSourceJSON.index","type":"index-pattern"}],"type":"visualization","updated_at":"2024-07-18T13:30:50.774Z","version":"WzEyNiwxXQ=="}
{"exportedCount":1,"missingRefCount":0,"missingReferences":[]}

What solution would you like?

I would like support for NDJSON file along with JSON to be added. Because we cannot use JSON for our use case

What alternatives have you considered?

As of now there is no other alternative I considered

n6809 commented 1 month ago

Hi, Any help about this topic would be highly appreciated.

dblock commented 1 month ago

Looks like a good feature request. Would love a PR with an implementation @n6809.

[Catch All Triage - 1, 2, 3]

opensearch-project / terraform-provider-opensearch