Improve the diff display in the Terraform provider's output when using `keep_original_source = true`

[aballiet]

Expected Behavior

We expect to have a proper diff when applying changes to the content of a flow.

Actual Behaviour

Currently output is unreadable.

Steps To Reproduce

1. Create a flow, like this one :

resource "kestra_flow" "example" {
  namespace            = "dev"
  flow_id              = "my-flow"
  keep_original_source = true
  content              = <<EOT
id: "my-flow"
namespace: "dev"
inputs:
  - name: my-value
    type: STRING

variables:
  first: "2"

tasks:
  - id: t2
    type: io.kestra.core.tasks.debugs.Echo
    format: first {{task.id}}
    level: TRACE

taskDefaults:
  - type: io.kestra.core.tasks.debugs.Echo
    values:
      format: third {{flow.id}}

EOT
}

1. Apply your flow (create it on your Kestra instance)
2. Change whatever row
3. terraform plan and check diff
4. You should have something like this :

You can reproduce the experience with keep_original_source = false. In that case we will work as expected.

Environment Information

• Kestra Version: 0.12 SNPASHOT
• Operating System (OS / Docker / Kubernetes): k8s
• Java Version (If not docker):

Example flow

No response

[aballiet]

Here is a loom video showcasing the issue: https://www.loom.com/share/c498cb2085e542c89070b00e067a0c44?sid=dbd0ea7c-a50c-4887-a64d-20b55c7f7f7c

Please ensure it is resolved before removing the option keep_original_source = false

[aballiet]

After digging a bit with @tchiotludo, this issue should be fixed by following using the CustomizeDiff function in the Terraform SDK to normalize and compare YAML content. This approach ensures consistent key ordering and structure during diff computations.

Here’s how you can implement this:

• Parse and Normalize YAML: Use a YAML parsing library (e.g., gopkg.in/yaml.v3 or similar) to parse the provided content string into a structured representation (like a map). Then, serialize it back to YAML or JSON with consistent key ordering.
• Implement CustomizeDiff: Read the content attribute from the planned resource configuration. Normalize the YAML content. Compare it against the current state to determine if changes exist.

We can leverage setAttribute function to do it, see doc here

ChatGPT code:

package provider

import (
    "bytes"
    "github.com/hashicorp/terraform-plugin-framework/datasource"
    "github.com/hashicorp/terraform-plugin-framework/resource"
    "github.com/hashicorp/terraform-plugin-framework/types"
    "gopkg.in/yaml.v3"
    "sort"
)

func normalizeYAML(input string) (string, error) {
    var parsed map[string]interface{}

    // Parse YAML
    if err := yaml.Unmarshal([]byte(input), &parsed); err != nil {
        return "", err
    }

    // Sort and serialize
    normalized, err := marshalSorted(parsed)
    if err != nil {
        return "", err
    }

    return normalized, nil
}

func marshalSorted(data interface{}) (string, error) {
    var buffer bytes.Buffer

    // Custom sort implementation for YAML keys
    encoder := yaml.NewEncoder(&buffer)
    defer encoder.Close()

    if err := encodeWithSortedKeys(encoder, data); err != nil {
        return "", err
    }

    return buffer.String(), nil
}

func encodeWithSortedKeys(encoder *yaml.Encoder, data interface{}) error {
    switch v := data.(type) {
    case map[string]interface{}:
        keys := make([]string, 0, len(v))
        for k := range v {
            keys = append(keys, k)
        }
        sort.Strings(keys)

        ordered := make(map[string]interface{})
        for _, k := range keys {
            ordered[k] = v[k]
        }
        return encoder.Encode(ordered)

    case []interface{}:
        for _, item := range v {
            if err := encodeWithSortedKeys(encoder, item); err != nil {
                return err
            }
        }
    default:
        return encoder.Encode(v)
    }
    return nil
}

func CustomizeDiff(ctx resource.CustomizeDiffContext, req resource.CustomizeDiffRequest, resp *resource.CustomizeDiffResponse) {
    var content types.String

    // Read the "content" attribute
    req.Config.GetAttribute(ctx, path.Root("content"), &content)
    if content.IsNull() || content.IsUnknown() {
        return
    }

    normalized, err := normalizeYAML(content.ValueString())
    if err != nil {
        resp.Diagnostics.AddError("YAML Normalization Error", "Failed to normalize YAML: "+err.Error())
        return
    }

    // Store normalized content as a computed attribute or compare it for diffs
    resp.Plan.SetAttribute(ctx, path.Root("normalized_content"), types.StringValue(normalized))
}