Passing complex objects to the tf.plan(tf_vars=...) doesn't work

[MrImpossibru]

Reopening the issue https://github.com/GoogleCloudPlatform/terraform-python-testing-helper/issues/67

Hello,

I've met an issue with passing complex variables to the tf.plan.

That doesn't work: tf.plan(output = True, tf_vars = { "pep_spec": { "pep1": "test" } }) because of: cmd_args = ('-no-color', '-input=false', '-var', "pep_spec={'pep1': 'test'}", ...)

Error is: "Single quotes are not valid. Use double quotes (") to enclose strings."

That works: tf.plan(output = True, tf_vars = { "pep_spec": "{\"pep1\"=\"test\"}" }) and that: tf.plan(output = True, tf_vars = { "pep_spec": "{\"pep1\":\"test\"}" }) but that is unusable.

The example is simplified. Passing a proper dictionary is required if I need to compare the value from the plan with the value from a dictionary to check if the value passes correctly.

Possible Solution is: Lines 147-149 of the tftest.py: cmd_args += list(itertools.chain.from_iterable( ("-var", "{}={}".format(k, json.dumps(v) if isinstance(v, (dict, list)) else v)) for k, v in tf_vars.items() ))

Terraform documentation about that - https://developer.hashicorp.com/terraform/language/values/variables#variables-on-the-command-line

@ludoo

So, usage in your case should be: tf.plan(output = True, pep_spec='{ "pep1": "test" }')

That is just a workaround like I sent above, not a solution. That won’t be possible to compare the equality of some field inside the dictionary with the value from the plan. Since there’s no proper handling I consider that behaviour as a bug anyway. Please, consider to fix that.

[MrImpossibru]

@ludoo The whole idea is to make tests like that:

import pytest
import tftest

@pytest.fixture(scope='session')
def vars() -> dict:
  return {
    "vnt_name": "vnet",
    "pep_spec": { "pep1": { "name": "test1" }, "pep2": { "name": "test2" } }
  }

@pytest.fixture(scope='session')
def plan(vars):
  tf = tftest.TerraformTest('Storageacc', './fixtures/')
  tf.setup()
  return tf.plan(output = True, tf_vars = vars)

def test_variables(vars, plan):
  for pep in vars['pep_spec']:
    assert plan.root_module.resources['azurerm_private_endpoint.private_endpoint["' + pep + '"]']['values']['name'] == vars['pep_spec'][pep]['name']

[ludoo]

  return tf.plan(output = True, **vars)

[MrImpossibru]

@ludoo

import pytest
import tftest

@pytest.fixture(scope='session')
def vars() -> dict:
  return {
    "vnt_name": "vnet",
    "pep_spec": { "pep1": { "name": "test1" }, "pep2": { "name": "test2" } }
  }

@pytest.fixture(scope='session')
def plan(vars):
  tf = tftest.TerraformTest('Storageacc', './fixtures/')
  tf.setup()
  return tf.plan(output = True, **vars)

def test_variables(vars, plan):
  for pep in vars['pep_spec']:
    assert plan.root_module.resources['azurerm_private_endpoint.private_endpoint["' + pep + '"]']['values']['name'] == vars['pep_spec'][pep]['name']

tftest:tftest.py:542 Error running command plan: 1  
Error: No value for required variable

  on variables.tf line 1:
   1: variable "vnt_name" {

The root module input variable "vnt_name" is not set, and has no default
value. Use a -var or -var-file command line argument to provide a value for
this variable.

Error: No value for required variable

  on variables.tf line 28:
  28: variable "pep_spec" {

The root module input variable "pep_spec" is not set, and has no default
value. Use a -var or -var-file command line argument to provide a value for
this variable.

[MrImpossibru]

@ludoo I can provide the whole example's code if necessary

[ludoo]

Gah started reproing and was distracted but I'm pretty sure I know what's happening: each var value needs to be the string representation of the HCL value, not Python literal code.

[ludoo]

@pytest.fixture(scope='session')
def vars() -> dict:
  return {
    "vnt_name": "vnet",
    "pep_spec": '{ pep1 = { name = "test1" }, pep2 = { name= "test2" } }'
  }

@pytest.fixture(scope='session')
def plan(vars):
  tf = tftest.TerraformTest('Storageacc', './fixtures/')
  tf.setup()
  return tf.plan(output = True, tf_vars=vars)

Sorry for misleading you, was thinking of the fixtures we use in the fabric repo :)

[MrImpossibru]

@ludoo yep, that is a workaround, but it won't solve the case. It will be hard to query and compare some internals from the string. Is it possible to implement what I proposed and make it just work? Then additionally to the workaround, using arrays and nested dictionaries will also work. (And I checked that)

[ludoo]

I don't think we want to change the way we do this, it would also involve converting Python objects to TF format, and some of those are really complex.

[MrImpossibru]

@ludoo Terraform accepts json format there. Like that: tf.plan(output = True, tf_vars = { "pep_spec": "{\"pep1\":\"test\"}" })

What I'm proposing is an extension, not a change. There's a condition to encode params as json only if isinstance(v, (dict, list)) what just doesn't work right now.

There will be no breaking changes.

[ludoo]

There will be no breaking changes.

Can you send a PR? And thanks for being persistent despite my attempts at discouraging / sidetracking. :)

[MrImpossibru]

@ludoo

Can you send a PR?

Done.

And thanks for being persistent despite my attempts at discouraging / sidetracking. :)

You're welcome! My team depends on that feature.

[ludoo]

Fixed by #69 will close once the new release is cut and pushed on pypi

[ludoo]

Thanks again for the change!

https://pypi.org/project/tftest/1.8.2/

[MrImpossibru]

@ludoo Thanks for your support!