scottwinkler / terraform-provider-shell

Terraform provider for executing shell commands and saving output to state file
Mozilla Public License 2.0
279 stars 61 forks source link

Feature request: add support for complex JSON structures #31

Closed bentterp closed 4 years ago

bentterp commented 4 years ago

I think it would be highly useful if the JSON structures could be more complex, ie contain nested maps and lists.

A real world example: az provider list --subscription ######################### | jq '.[]|select(.namespace == "Microsoft.RedHatOpenShift")' Output looks like this:

{
  "authorizations": [
    {
      "applicationId": "#########################",
      "managedByAuthorization": {
        "allowManagedByInheritance": true
      },
      "managedByRoleDefinitionId": "########################",
      "roleDefinitionId": "###########################"
    }
  ],
  "id": "/subscriptions/####################/providers/Microsoft.RedHatOpenShift",
  "namespace": "Microsoft.RedHatOpenShift",
  "registrationPolicy": "RegistrationRequired",
  "registrationState": "Registered",
  "resourceTypes": [
    {
      "aliases": null,
      "apiVersions": [
        "2019-12-31-preview"
      ],
      "capabilities": "None",
      "locations": [],
      "properties": null,
      "resourceType": "operations"
    }
  ]
}

I t would probably be a huge amount of work to implement parsing of that structure..... but I don't think you need to as Hashicorp has already done that: https://www.terraform.io/docs/configuration/functions/jsondecode.html

Running that function on the output above ( output "json_namespace" { value = jsondecode(file("./scripts/namespace.json")) } ) results in this variable

json_namespace = {
  "authorizations" = [
    {
      "applicationId" = "##########################"
      "managedByAuthorization" = {
        "allowManagedByInheritance" = true
      }
      "managedByRoleDefinitionId" = "##############################"
      "roleDefinitionId" = "################################"
    },
  ]
  "id" = "/###################################/providers/Microsoft.RedHatOpenShift"
  "namespace" = "Microsoft.RedHatOpenShift"
  "registrationPolicy" = "RegistrationRequired"
  "registrationState" = "Registered"
  "resourceTypes" = [
    {
      "apiVersions" = [
        "2019-12-31-preview",
      ]
      "capabilities" = "None"
      "locations" = []
      "resourceType" = "operations"
    },
  ]
}

which can then be used like any other map structure in the rest of the code.

scottwinkler commented 4 years ago

This is a great idea. I will see about implementing this

Note: I wonder if it would be possible to set the element type to *resource, using a custom node element. That way you could do something like: data.shell_script.test.state["key1"].getAsJSONelement["nested_key"].getAsString or something like that

scottwinkler commented 4 years ago

@bentterp I added support for complex nested JSON. Let me know what you think. If you like it then I will close this ticket.I still would like to consider a way to use custom nodes to access complex data, but I will just open a new ticket for that

scottwinkler commented 4 years ago

Closing this ticket since the feature was added to the latest release. I opened a new ticket to add support for traversing complex JSON based on what I suggested here

bentterp commented 4 years ago

@scottwinkler Great work, thanks.

I've been doing some testing here and it works in some cases, like if the output is a map, then I can pick attributes from the map. Lists seems to be a problem, but I'm not sure if that's due to Azure CLI or the parsing in the provider. IF I manage to create a good test for this and it fails, I'll open a new ticket.