Launchpad chokes on Terraform's output (AWS examples)

[cognomaton]

Reference documentation: https://github.com/Mirantis/launchpad/blob/master/examples/terraform/aws/README.md

The example states to run Launchpad as follows: terraform output mke_cluster | launchpad apply --config - The problem is that Terraform's output contains EOT markers:

user@machine:~/launchpad/examples/terraform/aws$ terraform output mke_cluster 
<<EOT
"apiVersion": "launchpad.mirantis.com/mke/v1.1"
"kind": "mke"
"spec":
  "hosts":
[snip]
...
[snip]
EOT
user@machine:~/launchpad/examples/terraform/aws$ 

Launchpad does not like the EOT markers.

A workaround is:

1. terraform output mke_cluster | tee launchpad.yaml
2. edit launchpad.yaml and remove both EOT lines
3. launchpad apply

[ankolesnikov]

1+2 can be done by: terraform output mke_cluster | sed '1d;$d' > ./launchpad.yaml or terraform output -json | yq e -P '.mke_cluster.value' - > ./launchpad.yaml

[cognomaton]

@ankolesnikov , thanks for the shortcuts.

The point is that the documentation is inaccurate at its current state, and thus needs to be updated. Any of the workarounds could be used to update the README.md; pick the one you like most.

[kke]

I wonder where this problem came from, possibly a new version of Terraform. It used to work fine.

I found this issue from terraforms tracker: https://discuss.hashicorp.com/t/terraform-outputs-with-heredoc-syntax-leaves-eot-in-file/18584/8 but there's no clear resolution.

The json output would be readable by launchpad (yaml reader can read json), but I don't think it's a good solution.

Maybe terraform output mke_cluster | grep -v EOT | launchpad apply --config - would be the simplest.

Need to investigate the options a little.

[kke]

Also the yaml output puts ugly quotes around everything, which would be nice to avoid.

[kke]

Maybe use a template? https://www.terraform.io/docs/language/functions/templatefile.html

[kke]

Or maybe use a regex replace on yamlencode result 🤔 https://github.com/hashicorp/terraform/issues/23322#issuecomment-716627898

[kke]

Appears this was changed in terraform 0.14:

The yamlencode function returns a string, which is then displayed by terraform output as a heredoc because it has multiple lines. We changed the format of terraform output in 0.14 to make it clearer what type values were, and to allow the output value to be used directly in HCL.

[kke]

There's --raw:

terraform output --raw mke_cluster ==>

"apiVersion": "launchpad.mirantis.com/mke/v1.3"
"kind": "mke"
"spec":
  "hosts":
  - "privateInterface": "ens5"
    "role": "manager"
    "ssh":
      "address": "xx.xx.102.33"
      "keyPath": "./ssh_keys/my-mke-cluster.pem"
      "user": "ubuntu"
  - "privateInterface": "ens5"
    "role": "worker"
    "ssh":
      "address": "xx.xx.62.141"
      "keyPath": "./ssh_keys/my-mke-cluster.pem"
      "user": "ubuntu"
  "mke":
    "adminPassword": "orcaorcaorca"
    "adminUsername": "admin"
    "installFlags":
    - "--default-node-orchestrator=kubernetes"
    - "--san=my-mke-cluster-master-lb-xxxxx.elb.eu-central-1.amazonaws.com"
  "msr": {}

[cognomaton]

@kke , perhaps the --raw option is the better one? It only requires a minor edit to the README.md. That'd be my vote, but I'll let the maintainer of the file/code/project make the choice.

[ankolesnikov]

yeah, with the latest terraform the outputs looks much better terraform output --raw mke_cluster ==>

apiVersion: launchpad.mirantis.com/mke/v1.3
kind: mke+msr
metadata:
  name: launchpad-mke
spec:
  hosts:
  - role: manager
    # imageDir: ./ucp_images
    hooks:
      apply:
        before:
          - ls -al > test.txt
        after:
          - cat test.txt
    ssh:
      address : 54.xx.xxx.13
      user    : ubuntu
      port    : 22
      keyPath : ./ssh_keys/xxx-mke-cluster.pem
    privateInterface : ens5
    # environment:
    #   http_proxy: http://example.com
    #   NO_PROXY: 10.0.0.*
    mcrConfig:
      debug: false
      log-opts:
        max-size: 10m
        max-file: "3"
  - role: manager
    # imageDir: ./ucp_images
    hooks:
      apply:
        before:
          - ls -al > test.txt
        after:
          - cat test.txt
    ssh:
      address : 18.xxx.xx.32
      user    : ubuntu
      port    : 22
      keyPath : ./ssh_keys/xxx-mke-cluster.pem
    privateInterface : ens5
    # environment:
    #   http_proxy: http://example.com
    #   NO_PROXY: 10.0.0.*
    mcrConfig:
      debug: false
      log-opts:
        max-size: 10m
        max-file: "3"
  - role: manager
    # imageDir: ./ucp_images
    hooks:
      apply:
        before:
          - ls -al > test.txt
        after:
          - cat test.txt
    ssh:
      address : 18.xxx.xx.180
      user    : ubuntu
      port    : 22
      keyPath : ./ssh_keys/xxx-mke-cluster.pem
    privateInterface : ens5
    # environment:
    #   http_proxy: http://example.com
    #   NO_PROXY: 10.0.0.*
    mcrConfig:
      debug: false
      log-opts:
        max-size: 10m
        max-file: "3"

  - role             : msr
    # imageDir: ./dtr_images
    hooks:
      apply:
        before:
          - ls -al > test.txt
        after:
          - cat test.txt
    ssh:
      address : 3.xxx.xx.241
      user    : ubuntu
      port    : 22
      keyPath : ./ssh_keys/xxx-mke-cluster.pem
    privateInterface : ens5
    # environment:
    #   http_proxy: http://example.com
    #   NO_PROXY: 10.0.0.*
    mcrConfig:
      debug: false
      log-opts:
        max-size: 10m
        max-file: "3"
  - role             : msr
    # imageDir: ./dtr_images
    hooks:
      apply:
        before:
          - ls -al > test.txt
        after:
          - cat test.txt
    ssh:
      address : 3.xxx.xx.63
      user    : ubuntu
      port    : 22
      keyPath : ./ssh_keys/xxx-mke-cluster.pem
    privateInterface : ens5
    # environment:
    #   http_proxy: http://example.com
    #   NO_PROXY: 10.0.0.*
    mcrConfig:
      debug: false
      log-opts:
        max-size: 10m
        max-file: "3"
  - role             : msr
    # imageDir: ./dtr_images
    hooks:
      apply:
        before:
          - ls -al > test.txt
        after:
          - cat test.txt
    ssh:
      address : 18.xxx.xx.197
      user    : ubuntu
      port    : 22
      keyPath : ./ssh_keys/xxx-mke-cluster.pem
    privateInterface : ens5
    # environment:
    #   http_proxy: http://example.com
    #   NO_PROXY: 10.0.0.*
    mcrConfig:
      debug: false
      log-opts:
        max-size: 10m
        max-file: "3"

  mke:
    version         : 3.3.3
    adminPassword   : xxxxxxxxx
    installFlags    :
    - --default-node-orchestrator=kubernetes
    - --san=xxx.amazonaws.com
    upgradeFlags    :
    - --force-minimums
    - --force-recent-backups
    licenseFilePath : xxxdocker_subscription_NFR.lic
    # configFile: ./mke-config.toml
    # configData: |-
    #   [scheduling_configuration]
    #     default_node_orchestrator = "kubernetes"
    # cloud:
    #   provider: azure
    #   configFile: ~/cloud-provider.conf
    #   configData: |-
    #     [Global]
    #     region=RegionOne
    # swarmInstallFlags
    # swarmUpdateCommands

  msr:
    version      : 2.7.4
    installFlags :
    - --ucp-insecure-tls
    - --dtr-external-url xxx.amazonaws.com
    upgradeFlags: []
    # replicaIDs: sequential | random

  mcr:
    version : 19.03.8
    channel: stable
    repoURL: https://repos.mirantis.com
    installURLLinux: https://get.mirantis.com/
    installURLWindows: https://get.mirantis.com/install.ps1

  cluster:
    prune: true