GCP compute metadata being overwrote

[ghost]

Hello,

I'm creating a CLI that will be used to test puppet code at my company. The cli creates windows and linux VMs deploys puppet code gets the status and then destroys them. I'm having trouble passing unique information to the gcp compute resource.

End users have a config script they pass the CLI and it is used throughout the code.

Here is the code for creating the config for gcp

if provider == "google":
                # Variables
                account_file = config_data["credentials"]
                instance_defaults = config_data["instance_defaults"]

                # Provider
                config += terrascript.provider.google(
                    credentials=f'${{file("{account_file}")}}',
                    project=config_data["project"])
                for image in images:
                    # Variables
                    hostname = "pup" + str(uuid.uuid4())[:8]
                    fqdn = hostname + ".domain.com"
                    i_name, i_family, i_project = image.values()
                    hosts.append([fqdn, i_family])
                    metadata = _GetMetadata(hostname, instance_defaults)

                    # Image Resource
                    image = terrascript.data.google_compute_image(
                        i_name, family=i_family, project=i_project)
                    config += image

                    # TF Variables
                    config += terrascript.Variable(hostname, default=fqdn)

                    # File Template
                    config += terrascript.data.template_file(
                        hostname,
                        template=f'${{file("{startup_script}")}}',
                        vars={
                            "environment": "${var.environment}",
                            "fqdn": f"${{var.{hostname}}}"
                        })

                    # Instance Resource
                    config += terrascript.resource.google_compute_instance(
                        hostname,
                        name=hostname,
                        hostname=fqdn,
                        machine_type="n1-standard-1",
                        zone=instance_defaults["zone"],
                        metadata=metadata,
                        boot_disk={
                            "initialize_params": {
                                "image": f"${{{image.self_link}}}"
                            }
                        },
                        network_interface=instance_defaults[
                            "network_interface"])

The issue i am coming across is the dynamic metadata i'm passing to the GCP compute resource is using the same values despite being passed unique values. The value it uses is the one passed to the last image.

the metadata is created using this fucntion

def _GetMetadata(hostname, instance_defaults):
    if instance_defaults.get("metadata") is not None:
        instance_defaults["metadata"].update({
            "windows-startup-script-ps1":
            f"${{data.template_file.{hostname}.rendered}}"
        })
        metadata = instance_defaults["metadata"]
    else:
        metadata = {
            "windows-startup-script-ps1":
            f"${{data.template_file.{hostname}.rendered}}"
        }
    return metadata

This is where i call the function

metadata = _GetMetadata(hostname, instance_defaults)

This is an example output i get back from the function, which is exactly what i expect (this example we are building 3 vms so the fucntion ran 3 seperate times each outputing expected values)

{'windows-startup-script-ps1': '${data.template_file.pup88a60312.rendered}'}
{'windows-startup-script-ps1': '${data.template_file.pup0a2d64f5.rendered}'}
{'windows-startup-script-ps1': '${data.template_file.pup66f88be2.rendered}'}

Note how the windows-startup-script-ps1 all reference unique template values.

However the JSON that is created from the above code is wrong. All gcp compute resource reference the same metadata.

To show what this looks like i printed the config after each iteration over the loop.

This example is creating 3 windows machines so the config is built over 3 loops then wrote to a file at the very end.

First iteration metadata =

{'windows-startup-script-ps1': '${data.template_file.pup5bbf3172.rendered}'}

The config looks like this

{
  "variable": {
    "environment": {
      "default": "puppet_env"
    },
    "pup5bbf3172": {
      "default": "pup5bbf3172.domain.com"
    }
  },
  "provider": {
    "google": [
      {
        "credentials": "${file(\"./secrets/account.json\")}",
        "project": "project"
      }
    ]
  },
  "data": {
    "google_compute_image": {
      "w2012": {
        "family": "windows-2012-r2",
        "project": "gce-uefi-images"
      }
    },
    "template_file": {
      "pup5bbf3172": {
        "template": "${file(\"./data/startup.ps1.tpl\")}",
        "vars": {
          "environment": "${var.environment}",
          "fqdn": "${var.pup5bbf3172}"
        }
      }
    }
  },
  "resource": {
    "google_compute_instance": {
      "pup5bbf3172": {
        "name": "pup5bbf3172",
        "hostname": "pup5bbf3172.domain.com",
        "machine_type": "n1-standard-1",
        "zone": "us-east4-c",
        "metadata": {
          "windows-startup-script-ps1": "${data.template_file.pup5bbf3172.rendered}"
        },
        "boot_disk": {
          "initialize_params": {
            "image": "${data.google_compute_image.w2012.self_link}"
          }
        },
        "network_interface": {
          "network": "network1",
          "subnetwork": "subnetwork2"
        }
      }
    }
  }
}

Everything above is exactly what i would expect.

The Second iteration has the following metadata: {'windows-startup-script-ps1': '${data.template_file.pup0fb6893d.rendered}'}

The output from the iteration is:

{
  "variable": {
    "environment": {
      "default": "puppet_env"
    },
    "pup5bbf3172": {
      "default": "pup5bbf3172.domain.com"
    },
    "pup0fb6893d": {
      "default": "pup0fb6893d.domain.com"
    }
  },
  "provider": {
    "google": [
      {
        "credentials": "${file(\"./secrets/account.json\")}",
        "project": "project"
      }
    ]
  },
  "data": {
    "google_compute_image": {
      "w2012": {
        "family": "windows-2012-r2",
        "project": "gce-uefi-images"
      },
      "w2016": {
        "family": "windows-2016",
        "project": "gce-uefi-images"
      }
    },
    "template_file": {
      "pup5bbf3172": {
        "template": "${file(\"./data/startup.ps1.tpl\")}",
        "vars": {
          "environment": "${var.environment}",
          "fqdn": "${var.pup5bbf3172}"
        }
      },
      "pup0fb6893d": {
        "template": "${file(\"./data/startup.ps1.tpl\")}",
        "vars": {
          "environment": "${var.environment}",
          "fqdn": "${var.pup0fb6893d}"
        }
      }
    }
  },
  "resource": {
    "google_compute_instance": {
      "pup5bbf3172": {
        "name": "pup5bbf3172",
        "hostname": "pup5bbf3172.domain.com",
        "machine_type": "n1-standard-1",
        "zone": "us-east4-c",
        "metadata": {
          "windows-startup-script-ps1": "${data.template_file.pup0fb6893d.rendered}"
        },
        "boot_disk": {
          "initialize_params": {
            "image": "${data.google_compute_image.w2012.self_link}"
          }
        },
        "network_interface": {
          "network": "network1",
          "subnetwork": "subnetwork2"
        }
      },
      "pup0fb6893d": {
        "name": "pup0fb6893d",
        "hostname": "pup0fb6893d.domain.com",
        "machine_type": "n1-standard-1",
        "zone": "us-east4-c",
        "metadata": {
          "windows-startup-script-ps1": "${data.template_file.pup0fb6893d.rendered}"
        },
        "boot_disk": {
          "initialize_params": {
            "image": "${data.google_compute_image.w2016.self_link}"
          }
        },
        "network_interface": {
          "network": "network1",
          "subnetwork": "subnetwork2"
        }
      }
    }
  }
}

As you can see it creates the variables correctly, the template_file correctly, but when it creates the gcp instance for some reason it overwrites all existing values for metadata with the new value of metadata...

You can see it happen again on the 3rd iteration every server will be updated to use the latest value for the metadata instead of just adding another gcp compute resource with the current value for metadata.

metadate for iteration 3: {'windows-startup-script-ps1': '${data.template_file.pup6e7889ba.rendered}'}

{
  "variable": {
    "environment": {
      "default": "puppet_env"
    },
    "pup5bbf3172": {
      "default": "pup5bbf3172.domain.com"
    },
    "pup0fb6893d": {
      "default": "pup0fb6893d.domain.com"
    },
    "pup6e7889ba": {
      "default": "pup6e7889ba.domain.com"
    }
  },
  "provider": {
    "google": [
      {
        "credentials": "${file(\"./secrets/account.json\")}",
        "project": "project"
      }
    ]
  },
  "data": {
    "google_compute_image": {
      "w2012": {
        "family": "windows-2012-r2",
        "project": "gce-uefi-images"
      },
      "w2016": {
        "family": "windows-2016",
        "project": "gce-uefi-images"
      },
      "w2019": {
        "family": "windows-2019",
        "project": "gce-uefi-images"
      }
    },
    "template_file": {
      "pup5bbf3172": {
        "template": "${file(\"./data/startup.ps1.tpl\")}",
        "vars": {
          "environment": "${var.environment}",
          "fqdn": "${var.pup5bbf3172}"
        }
      },
      "pup0fb6893d": {
        "template": "${file(\"./data/startup.ps1.tpl\")}",
        "vars": {
          "environment": "${var.environment}",
          "fqdn": "${var.pup0fb6893d}"
        }
      },
      "pup6e7889ba": {
        "template": "${file(\"./data/startup.ps1.tpl\")}",
        "vars": {
          "environment": "${var.environment}",
          "fqdn": "${var.pup6e7889ba}"
        }
      }
    }
  },
  "resource": {
    "google_compute_instance": {
      "pup5bbf3172": {
        "name": "pup5bbf3172",
        "hostname": "pup5bbf3172.domain.com",
        "machine_type": "n1-standard-1",
        "zone": "us-east4-c",
        "metadata": {
          "windows-startup-script-ps1": "${data.template_file.pup6e7889ba.rendered}"
        },
        "boot_disk": {
          "initialize_params": {
            "image": "${data.google_compute_image.w2012.self_link}"
          }
        },
        "network_interface": {
          "network": "network1",
          "subnetwork": "subnetwork2"
        }
      },
      "pup0fb6893d": {
        "name": "pup0fb6893d",
        "hostname": "pup0fb6893d.domain.com",
        "machine_type": "n1-standard-1",
        "zone": "us-east4-c",
        "metadata": {
          "windows-startup-script-ps1": "${data.template_file.pup6e7889ba.rendered}"
        },
        "boot_disk": {
          "initialize_params": {
            "image": "${data.google_compute_image.w2016.self_link}"
          }
        },
        "network_interface": {
          "network": "network1",
          "subnetwork": "subnetwork2"
        }
      },
      "pup6e7889ba": {
        "name": "pup6e7889ba",
        "hostname": "pup6e7889ba.domain.com",
        "machine_type": "n1-standard-1",
        "zone": "us-east4-c",
        "metadata": {
          "windows-startup-script-ps1": "${data.template_file.pup6e7889ba.rendered}"
        },
        "boot_disk": {
          "initialize_params": {
            "image": "${data.google_compute_image.w2019.self_link}"
          }
        },
        "network_interface": {
          "network": "network1",
          "subnetwork": "subnetwork2"
        }
      }
    }
  }
}

At this point the config is wrote to a file and terraform is ran.

But the desired outcome wasn't reached every gcp compute resource now references the most recently created metadata object.