processing multiline tf-runner logs

[pbikki]

It appears that tf-runner logs are multi-line and don't have a specific format. When shipped to a logging service, multiline logs are not processed by default which makes it difficult to read thru. Is there a setting I'd need to use to render a diff format for json logs

TF controller version - v0.15.1

▶ kubectl logs -f test-tf-runner
2023/11/22 16:51:56 Starting the runner... version  sha
{"level":"info","ts":"2023-11-22T16:52:10.941Z","logger":"runner.terraform","msg":"preparing for Upload and Extraction","instance-id":""}
{"level":"info","ts":"2023-11-22T16:52:10.961Z","logger":"runner.terraform","msg":"setting envvars","instance-id":"9a349873-090e-45e8-b71b-012dcfe6c00a"}
{"level":"info","ts":"2023-11-22T16:52:10.961Z","logger":"runner.terraform","msg":"getting envvars from os environments","instance-id":"9a349873-090e-45e8-b71b-012dcfe6c00a"}
{"level":"info","ts":"2023-11-22T16:52:10.962Z","logger":"runner.terraform","msg":"setting up the input variables","instance-id":"9a349873-090e-45e8-b71b-012dcfe6c00a"}
{"level":"info","ts":"2023-11-22T16:52:10.962Z","logger":"runner.terraform","msg":"mapping the Spec.Values","instance-id":"9a349873-090e-45e8-b71b-012dcfe6c00a"}
{"level":"info","ts":"2023-11-22T16:52:10.962Z","logger":"runner.terraform","msg":"mapping the Spec.Vars","instance-id":"9a349873-090e-45e8-b71b-012dcfe6c00a"}
{"level":"info","ts":"2023-11-22T16:52:10.963Z","logger":"runner.terraform","msg":"mapping the Spec.VarsFrom","instance-id":"9a349873-090e-45e8-b71b-012dcfe6c00a"}
{"level":"info","ts":"2023-11-22T16:52:10.974Z","logger":"runner.terraform","msg":"generating the template founds"}
{"level":"info","ts":"2023-11-22T16:52:10.974Z","logger":"runner.terraform","msg":"main.tf.tpl not found, skipping"}
{"level":"info","ts":"2023-11-22T16:52:10.976Z","logger":"runner.terraform","msg":"initializing","instance-id":"9a349873-090e-45e8-b71b-012dcfe6c00a"}
{"level":"info","ts":"2023-11-22T16:52:10.976Z","logger":"runner.terraform","msg":"mapping the Spec.BackendConfigsFrom","instance-id":"9a349873-090e-45e8-b71b-012dcfe6c00a"}
{
  "terraform_version": "1.3.9",
  "platform": "linux_amd64",
  "provider_selections": {},
  "terraform_outdated": true
}
Upgrading modules...
Downloading git::https://github.com/oracle-terraform-modules/terraform-oci-oke.git?ref=239dad6 for oke...
- oke in .terraform/modules/oke
- oke.bastion in .terraform/modules/oke/modules/bastion
- oke.cluster in .terraform/modules/oke/modules/cluster
- oke.extensions in .terraform/modules/oke/modules/extensions
- oke.iam in .terraform/modules/oke/modules/iam
- oke.network in .terraform/modules/oke/modules/network
- oke.operator in .terraform/modules/oke/modules/operator
- oke.utilities in .terraform/modules/oke/modules/utilities
Downloading registry.terraform.io/oracle-terraform-modules/vcn/oci 3.5.3 for oke.vcn...
- oke.vcn in .terraform/modules/oke.vcn
- oke.vcn.subnet in .terraform/modules/oke.vcn/modules/subnet
- oke.workers in .terraform/modules/oke/modules/workers

Initializing the backend...

Successfully configured the backend "kubernetes"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding hashicorp/null versions matching ">= 3.2.1"...
- Finding oracle/oci versions matching ">= 4.67.3, >= 4.115.0"...
- Finding hashicorp/random versions matching ">= 3.4.3"...
- Finding hashicorp/time versions matching ">= 0.9.1"...
- Finding hashicorp/cloudinit versions matching ">= 2.2.0"...
- Finding hashicorp/http versions matching ">= 3.2.1"...
- Finding hashicorp/helm versions matching ">= 2.9.0"...
- Installing hashicorp/random v3.5.1...
- Installed hashicorp/random v3.5.1 (signed by HashiCorp)
- Installing hashicorp/time v0.9.1...
- Installed hashicorp/time v0.9.1 (signed by HashiCorp)
- Installing hashicorp/cloudinit v2.3.2...
- Installed hashicorp/cloudinit v2.3.2 (signed by HashiCorp)
- Installing hashicorp/http v3.4.0...
- Installed hashicorp/http v3.4.0 (signed by HashiCorp)
- Installing hashicorp/helm v2.11.0...
- Installed hashicorp/helm v2.11.0 (signed by HashiCorp)
- Installing hashicorp/null v3.2.2...
- Installed hashicorp/null v3.2.2 (signed by HashiCorp)
- Installing oracle/oci v5.21.0...
- Installed oracle/oci v5.21.0 (signed by a HashiCorp partner, key ID 1533A49284137CEB)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!
{"level":"info","ts":"2023-11-22T16:52:20.017Z","logger":"runner.terraform","msg":"workspace select"}
Workspace "<redacted>" already exists
{"level":"info","ts":"2023-11-22T16:52:21.970Z","logger":"runner.terraform","msg":"workspace new:, workspace \"<redacted>\" already exists"}
Switched to workspace "<redacted>".

[eatwithforks]

@chanwit is it possible to jsonify all of the output? This will make log forwarding to services more accessible and readable.

[chanwit]

Hi @eatwithforks

technically possible if it's allowed by the underlying binary.