Allow negative negative indices in slice()

[hamstah]

Current Terraform Version

Terraform v0.12.2

Use-cases

My use case is being able to replace a suffix in an AWS ARN for secrets manager The arn has the following structure

arn:aws:secretsmanager:us-east-1:123456789012:secret:path/to/secret-name-<random chars>

I want to replace the last 6 chars with a wildcard. Having the ability to use negative indices I could split the ARN on - then slice -1 as the end index to remove the last element, then concatenate with my wildcard.

This would allow editing of lists in place instead of having to compute the length to pass

Attempted Solutions

This works but is quite verbose and requires intermediate variables

locals {
  split_arn = split("-", aws_secretsmanager_secret.secret.id)
  arn_without_suffix = slice(local.split_arn, 0, length(local.split_arn)-1)
  wildcard_arn = join("-", concat(local.arn_without_suffix, ["??????"]))
}

We could transform it into a 1 liner by splitting twice (once for the value and once for the length) but that becomes unreadable.

Proposal

Allow negative indices for the end of the slice in the slice function

• Allow negative indices in https://github.com/hashicorp/terraform/blob/master/lang/funcs/collection.go#L1141
• If the index is negative, substract it from the list length in https://github.com/hashicorp/terraform/blob/master/lang/funcs/collection.go#L1117

Then have a look for other similar functions that should support it and make it consistent, for example element().

With this change the example above can be simplified to

join("-", concat(slice(split("-", aws_secretsmanager_secret.secret.id), 0, -1), ["??????"]))

References

https://github.com/hashicorp/terraform/issues/16044 https://github.com/hashicorp/terraform/issues/15582

[marshallford]

I'm also interested in this functionality.

[antonosmond]

This would be very useful. It'd also bring this function inline with the way the substr() function works currently which already allows a negative index.

[txomon]

The solution I reached was to use reverse() in the list, and use positive numbers.

[Satak]

Terraform should indeed have a negative index search to get the last element/index in list, as like this: var.my_list[-1]

Please add this!

Though it's quite clever (but not as clean and intuitive) to use the reverse function: reverse(var.my_list)[0]

[schollii]

Using reverse(list)[N-1] as a substitute for list[-N] is an anti-pattern as far as I'm concerned:

• It is verbose
• It is error prone: you have to convert indices if you want the second to last you could have list[-2] which matches your thought of second to last, whereas in the reversal you will need reverse(list)[1] where "second" is not as obvious
• it is waste of computation: a new list must be created and populated via a loop
• It obfuscates intent, leaving the reader to guess what the reversal is for (likely the Nth element from the right, but is it for sure the intent? no).

I cannot think of a good reason to not support negative indices in all array-related functions, as the length of the list is always known to terraform.

Meanwhile I use the same approach as the Attempted Solution of the OP, name your variables clearly and anyone seeing the code knows exactly what is going on and how to modify it.

[txomon]

Using reverse(list)[N-1] as a substitute for list[-N] is an anti-pattern as far as I'm concerned

I agree, and would love if some author or contributor could have a look on it. I only wanted to provide a workaround for whoever stumbles into this thread and is looking for a possible solution

[schollii]

I only wanted to provide a workaround for whoever stumbles into this thread and is looking for a possible solution

The solution until then should be as suggested by the OP, since we've all seen how "temporary workarounds" almost always end up permanent in your code. The general pattern of OP, ie not just for replacing last item but other parts of string segmented by CHAR, is:

locals {
  split_var = split(CHAR, variable)
  sliced = slice(local.split_var, START, END)
  new_var = join(CHAR, expression involving local.sliced)
}

[zarnovican]

Your usecase can be workarounded with regex().

Replace last 6 chars

> "${regex("(.*).{6}$", "arn:aws:secretsmanager:us-east-1:123456789012:secret:path/to/secret-name-fbghts")[0]}??????"
"arn:aws:secretsmanager:us-east-1:123456789012:secret:path/to/secret-name-??????"

Replace everything after the last "-"

> "${regex("(.*-).*$", "arn:aws:secretsmanager:us-east-1:123456789012:secret:path/to/secret-name-fbghts")[0]}??????"
"arn:aws:secretsmanager:us-east-1:123456789012:secret:path/to/secret-name-??????"

Re: feature request

• :+1: for negative indexes
• :+1: for optional end index
• :+1: for negative indexes that can go below 0 and don't produce an error

Same semantic as in Python for example:

>>> ('a', 'b', 'c')[-2:-1]
('b',)
>>> ('a', 'b', 'c')[-2:]
('b', 'c')
>>> ('a', 'b', 'c')[-5:]
('a', 'b', 'c')
>>> 

[torbendury]

Did anything ever happen here?

[crw]

Thank you for your continued interest in this issue.

Terraform version 1.8 launches with support of provider-defined functions. It is now possible to implement your own functions! We would love to see this implemented as a provider-defined function.

Please see the provider-defined functions documentation to learn how to implement functions in your providers. If you are new to provider development, learn how to create a new provider with the Terraform Plugin Framework. If you have any questions, please visit the Terraform Plugin Development category in our official forum.

We hope this feature unblocks future function development and provides more flexibility for the Terraform community. Thank you for your continued support of Terraform!

[Satak]

@crw HashiCorp please make then an official function extension provider where the community can contribute.

For big enterprise companies, that has high security and quality standards, using some random guy’s from Nebraska function extension provider is really a no go.

[RussellRollins]

Wow, the state of Nebraska really catching strays in this one.

[AndrewJo]

Probably a reference to this xkcd comic:

[atykhyy]

For all that I am shocked that this issue exists and has been open for 5 years already, HCL explicitly rejected the idea of using negative numbers to index from the end of a list/tuple:

// Some other languages allow negative indices to
// select "backwards" from the end of the sequence,
// but HCL doesn't do that in order to give better
// feedback if a dynamic index is calculated
// incorrectly.

Hashi may be understandably reluctant to change this, because at this point, a lot of HCL code has been written assuming the above behavior and the change will be potentially breaking. Enabling negative indices in [] and functions like slice will make formerly incorrect HCL code suddenly start working and producing unexpected and unwanted results, or formerly correct HCL code will suddenly start producing different results (e.g. if using try(a_tuple[computed_index_expr], fallback_expr)). Adding new functions with new names for new behaviors seems like a reasonable alternative to running every slice and list index through an external function provider, not to mention having to type provider::js::index(...) and provider::js::slice(...) every time.

ETA: while the above-quoted comment which documents the thinking behind the behavior is only 3 years old, both the behavior in question and the language specification have been like that from the start. The relevant section of HCL specification says:

If the index operator is applied to a value of tuple or list type, the key expression must be an non-negative integer number representing the zero-based element index to access.

[olivers-xaxis]

@cpboyd as much as I would like that feature, I have to agree with @atykhyy that introducing this now would break existing code, in many cases silently. This is not acceptable. Yes languages evolve, but some fundamentals cannot change if they are going to break existing usages beyond your own code. One needs to either introduce a new function, a new operator, or new syntax so that the old behavior continues in existing code.

Eg we could introduce a new function: get_item(list, index) since this does not currently exist. It could therefore support a variety of additional behaviours such as: negative indices, default value (via an extra optional arg) if index >= length(list), overloading so get_item(map, key, default) also works, and there might be more.