The ability to split a long string over multiple lines

[nebffa]

When using a really long string like https://mystorageaccount.blob.core.windows.net/system/Microsoft.Compute/Images/windows-2016-datacenter/packer-osDisk.8c22742f-d22d-4f1a-babd-7712381c413e.vhd, it forces editors to horizontally scroll. I'd like the ability to split the definition of strings like this over multiple lines. Note that this is different than splitting text over multiple lines with:

text = <<EOF
Text
Here
EOF

[sethvargo]

Hi @nebffa

Thank you for opening an issue. I think this is a bit outside the scope of HCL. Most editors have text-wrapping functionality for long lines like this.

[mitchellh]

I think this will be possible in future changes coming to HCL. We're planning to more closely integrate HCL and HIL (the interpolation language used by Terraform) and in doing so it should be possible to do something like:

a = "foo" +
"bar"

And handle the newlines yourself. This won't be possible with HCL in its current form.

[nebffa]

Hey guys, thanks for your quick responses. Sounds good @mitchellh , it will make a big difference to readability whenever it lands down the track. Keep up the great work!

[mr-olson]

+1 This is also an issue for me for long commandline local-exec statements - without line wrapping in an editor, interesting things scroll way off, and incremental changes in source control are more tedious to follow.

(https://github.com/hashicorp/terraform/issues/8210) and then (https://github.com/hashicorp/hcl/issues/144) attempted to do this with HEREDOC syntax & line coalescing, but the approach mentioned doesn't work as noted in the second issue.

The suggested + continuation here is much tidier anyway.

[georgehdd]

Any updates?

[sandstrom]

Also curious, is this planned for HCL2?

Something like Python triple quotes or Ruby's squiggly heredoc that would understand indentation would be very useful.

my_dict {
  my_string = """indentation would be 'chopped off' 
                 equal to the point where the
                 triple-quote starts"""

my_dict {
  my_string = <<-HEREDOC
    or, alternatively, the indentation would be chopped off equal to
    the 'lowest' indentation within the block.
    (or some other rule that's even better)
  HEREDOC
}

[jerome-arzel]

I think the C way is the cleanest:

foo =
  "This is the first line. "
  "And there is the second"

[rb-cloud-guru]

I think this goes along my dsc node config statement:

resource "azurerm_automation_dsc_configuration" "lcm" { name = "TestConfig" resource_group_name = "${azurerm_resource_group.rg.name}" automation_account_name = "${azurerm_automation_account.dsc.name}" location = "${azurerm_resource_group.rg.location}" content_embedded = "configuration TestConfig{
Node IsWebServer { WindowsFeature IIS { Ensure = "Present" Name = "Web-Server" IncludeAllSubFeature = "$true" } } }"

At 48:5: literal not terminated [}"] is line 48

}

[rb-cloud-guru]

Heredoc syntax as pointed out by Tom

[apparentlymart]

Hi all,

Posting +1 here does nothing except create noise for those who are following this issue. Please don't do it. If you want to show your interest, add a :+1: reaction to the original comment instead.

(I have deleted all of the previous +1 comments.)

[choovick]

This might work on properties and fields that support functions (does not work for description):

output test {
  value = join("", [
    "This is the first line ",
    "and this is the second part of the same line"
  ])
}

but also interested in seeing syntax mentioned above

a = "foo" +
"bar"

[pared-brandon]

This issue affects module specification. Ideally, a module block could have separate parameters for the repository URL, the module path, and the ref parameter, all with sensible defaults. Instead, we have perhaps the worst format from a line-based diff perspective: everything crammed onto one long line with the most frequently changed thing at the end. Three-way diff on a 1080p display is unnecessarily stressful. Line-splitting would provide a workaround, but source as a block would be amazing.

[apparentlymart]

Hi @pared-brandon,

If I'm understanding correctly what you're referring to, that seems like Terraform feedback rather than HCL feedback because the module source argument is a Terraform concept. Would you mind opening a feature request for that over in the Terraform repository instead, so that the Terraform team can see it?

[pared-brandon]

Ah, yes. Thanks, @apparentlymart, will do. I'll reference this issue.

[ksvladimir]

A somewhat weird workaround that works today:

module "test" {
  source = "${
     "the first part"    }${
     "the second part"   }${
     "the last part"
  }"
}

[techdragon]

This is particularly problematic for variable validation in terraform modules. Writing meaningful error messages requires using long strings, and because of how variable definitions work, it's impossible to use any of the workarounds suggested so far in this issue. Its not uncommon for a sensible error message or description to exceed 200 characters.

Can we not just simplify this to a special case of parenthesis handling ?

really_long_string = (
    "This is the string that never ends. Yes, it goes on and on, my friend. "
    "Some people started typing it not knowing what it was, "
    "And they’ll continue typing it forever, just because..."
)

If we have parenthesis with strings inside it, with nothing else but whitespace between the strings, simply concatenate them. I've spent a few minutes thinking and I can't think of anywhere this new behaviour would mess up existing code, in terraform at least.

Since the original idea from @mitchellh has been significantly changed in HCL2 ... as explained by @apparentlymart here https://github.com/hashicorp/hcl/pull/171#issuecomment-613081509

For anyone else seeing this and wondering what happened: Integrating HIL with HCL (beyond what I started here) turned out to be pretty impractical without breaking API changes, and so this initial idea is what developed into HCL 2.0, now available as github.com/hashicorp/hcl/v2. HCL 2 incorporates a HIL-like expression language, effectively merging the two into a single language so that HIL as a separate entity is no longer needed.

While you can do string interpolation using HCL2 in most places you need to split a long string, variable definitions are not one of these places... So having some HCL native syntax that just means "this is only a string, not an interpolation, not a function, just a regular string, valid anywhere a string is valid", is important to avoid the need for changing how variables get defined.

Edit: I did just find these over in the terraform repo that would enable using join as a workaround in my highlighted case of a variable' error_message attribute , https://github.com/hashicorp/terraform/issues/24407 and a pull request to fix it in https://github.com/hashicorp/terraform/pull/25028 ... but I still think it would be good to have a long term fix for improving the developer ergonomics around the common case of "I need a long string" that wont require every use for a string parameter to accept functions/expressions instead.

[timblaktu]

Wait, does the openness of this Issue mean that this documenation and this documentation is wrong? These clearly state that HCL (<1.5) supports multiline string literals in the form of the heredoc syntax. I'm about to use this syntax for inline shell provisioner commands in packer.

[AndrewSav]

@timblaktu no, heredoc is different.

[timblaktu]

Hm. I see that the documentation I linked to says the heredoc support is only for defining "string literals", but isn't that exactly what this issue is asking for? For example, when I read those docs, I assumed that I could do this in my hcl2 packer template:

  provisioner "shell" {
    inline_shebang = "/bin/bash -el"
    inline = [
      "set -x",
      "echo provisioning machine built from commit ${var.commit}...",
      "ansible --version && which ansible",
      <<-EOF
      ansible-pull --directory /home/ansible/src/cm \
                   --checkout "feature/SWOPS-736" \
                   --url ssh://git@bitbucket:7999/tools/configuration-management.git \
                   --inventory localhost,
      EOF
    ]
  }

...and it just worked. What sorts of strings are you trying to split over multiple lines in HCL that are not working with heredoc? The examples I see in this issue appear identical to my use case above (or the use case presented in the doc links).

Perhaps this issue really means "Provide a more syntactically-elegant way to split a string over multiple lines than heredoc" ??

[AndrewSav]

...and it just worked

The way I see it it worked because bash understands \ as line continuation. But this issue is not specific to bash.

[corydodt]

Hm. I see that the documentation I linked to says the heredoc support is only for defining "string literals", but isn't that exactly what this issue is asking for? For example, when I read those docs, I assumed that I could do this in my hcl2 packer template:

@timblaktu the main difference between heredoc and the request being made in this case is that a heredoc inserts newlines. In other words, the string is multiline in the sense of containing newlines in its displayed form, and those newlines are wherever they fall in the heredoc. In your example, the placement of newlines does not break the result because bash is capable of parsing it with newlines.

For a case where heredoc does not work, consider the string I just had to write:

output "database_url" {
    value = "postgres://${aws_db_instance.default.username}:${aws_db_instance.default.password}@${aws_db_instance.default.endpoint}/${aws_db_instance.default.name}"
}

This string cannot contain newlines, so heredoc is not helping.

Note that I can (and did) use join() in this context, but wouldn't it be infinitely nicer to write this?

output "database_url" {
    value = "postgres://"
        "${aws_db_instance.default.username}:${aws_db_instance.default.password}@"
        "${aws_db_instance.default.endpoint}/"
        "${aws_db_instance.default.name}"
}

[bjtucker]

Still looking for this. Of course our editors can handle long lines, but they're ugly and less clear. FWIW, overloading + like @mitchellh suggested would be my favorite solution.

[jantari]

I'm essentially looking for the YAML operator >- (fold newlines into spaces and leave no trailing newline) but for HCL.

Other folks might also have usecases for "just" folding newlines without replacing them with spaces. Currently, I'd be OK with the space-joining that >- does.

[JoshM1994]

My solution was to use

join("", [
"string 1 that is really long",
"and then the second part of that long string"
])

Not hugely clean but very effective for my use-case (AWS IoT SQL string)

[lmayorga1980]

I've been trying to use Packer HCL with multi-line string but it is not working for me.

  metadata = {
    windows-startup-script-cmd = <<EOF
   ....
EOF
  }

Error: Invalid expression

  on gcp-win-2019-ansible.pkr.hcl line 39, in source "googlecompute" "windows-ssh-ansible":
  39:     windows-startup-script-cmd = <<EOF 

[mahvar]

Any update on this?

[olivierGobet]

Heredoc string works fine for me.

[dimisjim]

@olivierGobet

Heredoc would result in a multiline string, this issue is about defining a single line string in multiple lines.

[olivierGobet]

@dimisjim thanks for your intervention.

and what about doing something a bit like this?

locals {
    test = replace(
<<EOT
hello
world
EOT
    ,"/\r\n/", " ")
}

output test-output {
    value = local.test
}

Outputs:

test-output = "hello world "

[AndrewSav]

Is it \r\n or \r or \n or do we need a separate version for mac, linux and windows? What is the text has both long strings that we need to break up and end of line character?

[JBallin]

Another workaround: break up the string into variables.

Before

inputs = {
  url = "https://mystorageaccount.blob.core.windows.net/system/Microsoft.Compute/Images/windows-2016-datacenter/packer-osDisk.8c22742f-d22d-4f1a-babd-7712381c413e.vhd"
}

After

locals {
  domain = "https://mystorageaccount.blob.core.windows.net"
  file   = "packer-osDisk.8c22742f-d22d-4f1a-babd-7712381c413e.vhd"
  path   = "system/Microsoft.Compute/Images/windows-2016-datacenter"
}

inputs = {
  url = "${local.domain}/${local.path}/${local.file}"
}

[trallnag]

Here is the outright best workaround 😛. Use inline YAML that is decoded with Terraform:

locals {
  mystring = yamldecode(<<-EOT
    x: >-
      fgr
      grg
      gr
  EOT
  ).x
}

This will evaluate to "fgr grg gr".

This approach can be used everywhere, not just in locals blocks.

@jantari, if you want the YAML indicators, just use YAML.

[mahvar]

oh that's a nice trick - thank you!

On Tue, Jan 3, 2023 at 6:56 AM Tim Schwenke @.***> wrote:

Here is the outright best workaround 😛. Use inline YAML that is decoded with Terraform:

locals {

mystring = yamldecode(<<-EOT x: >- fgr grg gr EOT

).x

}

This will evaluate to "fgr grg gr"

— Reply to this email directly, view it on GitHub https://github.com/hashicorp/hcl/issues/211#issuecomment-1369861898, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH77W2ONM4QLYDFER4UEWYDWQQ43VANCNFSM4D3H65MA . You are receiving this because you commented.Message ID: @.***>

-- Fereshteh Mahvar | Staff Medical Device Software Engineer - Google Health | @.*** | +1 650-448-9653

[mgzenitech]

Here is the outright best workaround stuck_out_tongue. Use inline YAML that is decoded with Terraform:

locals {
  mystring = yamldecode(<<-EOT
    x: >-
      fgr
      grg
      gr
  EOT
  ).x
}

This will evaluate to "fgr grg gr".

This approach can be used everywhere, not just in locals blocks.

@jantari, if you want the YAML indicators, just use YAML.

Wow the lengths we have to take just to have simple inverse slash implemented :D

locals {
  mystring = "VERRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRYYYYYY \
             LONG LINEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE!!"
}

Or maybe just do what Python does:

locals {
  mystring = (
    "VERRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRYYYYYY"
    "LONG LINEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE!!"
  )
}

[blieusong]

In case it helps, another workaround that I use is the join function.

Example:

  description = join(" ", [
    "The five", 
    "boxing wizards",
    "jump quickly." 
  ])

[jantari]

@blieusong see this comment from 2 years ago: https://github.com/hashicorp/hcl/issues/211#issuecomment-974484775