libvirt_cloudinit_disk does not work on Terraform Cloud

[MadsRC]

System Information

Linux distribution

Latest version of Ubuntu LTS as mentioned here

Terraform version

1.1.5

Provider and libvirt versions

Latest available from Hashicorps Terraform Provider registry

---

Checklist

• [x] Is your issue/contribution related with enabling some setting/option exposed by libvirt that the plugin does not yet support, or requires changing/extending the provider terraform schema?

  • [x] Make sure you explain why this option is important to you, why it should be important to everyone. Describe your use-case with detail and provide examples where possible.
  • [x] If it is a very special case, consider using the XSLT support in the provider to tweak the definition instead of opening an issue
  • [x] Maintainers do not have expertise in every libvirt setting, so please, describe the feature and how it is used. Link to the appropriate documentation

• [x] Is it a bug or something that does not work as expected? Please make sure you fill the version information below:

Description of Issue/Question

Setup

terraform {
  required_providers {
    libvirt = {
      source = "dmacvicar/libvirt"
    }
  }
}

provider "libvirt" {
  uri = "qemu+ssh://virt:${var.ssh_pass}@THEIP/system?sshauth=ssh-password&knownhosts=./known_hosts"
}

locals {
    libvirt = {
        pools = {
            vm = {
                name = "virtualMachines"
            }
        }
    }
}

resource "libvirt_pool" "virtualMachines" {
    name = local.libvirt.pools.vm.name
    type = "dir"
    path = "/aPath/${local.libvirt.pools.vm.name}"
}

data "template_file" "user_data" {
  template = file("${path.module}/cloud_init.cfg")
}

data "template_file" "network_config" {
  template = file("${path.module}/network_config.cfg")
}

resource "libvirt_cloudinit_disk" "commoninit" {
  name           = "commoninit.iso"
  user_data      = data.template_file.user_data.rendered
  network_config = data.template_file.network_config.rendered
  pool           = libvirt_pool.virtualMachines.name
}

Steps to Reproduce Issue

Attempt to create a libvirt_cloudinit_disk resource using Terraform running on Hashicorp managed Terraform Cloud runners.

---

Additional information:

The call to an external dependency seems to happen here.

According to Terraform Cloud, one should not install additional tools on their runners - While it is supported, it should only be as a last resort. This is documented here and here.

While this does not make the provider entirely incompatible with Terraform Cloud, it sure does make it very tiresome to work with.

See thee attached screenshot for exact error message

[masterbender]

using 0.6.14 , i can not create any vm (vor example simple ubuntu cloudinit example) via terraform ( on premise server not cloud ) :

Error: error while starting the creation of CloudInit's ISO image: exec: "mkisofs": executable file not found in $PATH
│ 
│   with libvirt_cloudinit_disk.commoninit,
│   on main.tf line 8, in resource "libvirt_cloudinit_disk" "commoninit":
│    8: resource "libvirt_cloudinit_disk" "commoninit" {

[jbeisser]

I've also found this issue when using a MacOS system to build from.

@MadsRC's PR starts down the right path, but the problem about Joliet and RockRidge extensions aren't supported, and I believe are required for the iso9660 image. But, with a little bit of testing, I verified that vfat works for the localds image. Which is supported by go-diskfs, making that approach viable.

[axxyhtrx]

using 0.6.14 , i can not create any vm (vor example simple ubuntu cloudinit example) via terraform ( on premise server not cloud ) :

Error: error while starting the creation of CloudInit's ISO image: exec: "mkisofs": executable file not found in $PATH
│ 
│   with libvirt_cloudinit_disk.commoninit,
│   on main.tf line 8, in resource "libvirt_cloudinit_disk" "commoninit":
│    8: resource "libvirt_cloudinit_disk" "commoninit" {

try > brew install cdrtools

[jbeisser]

Using cdrtools doesn't solve the issue with Terraform Cloud though.

[skindud]

I use docker image based on Alpine for terraform running and it has been solved for me by installing mkisofs package by the following command: apk add cdrkit

[jseparovic]

@MadsRC Did you ever get this working on Terraform Cloud?

I've been trying the workaround for the last few hours to install a genisoimage (mkisofs) binary but I keep getting a exec format error:

Error: error while starting the creation of CloudInit's ISO image: fork/exec /home/tfc-agent/.tfc-agent/component/terraform/runs/run-ceETzGaMcHNTiXrf/.local/bin/mkisofs: exec format error

I've pulled the binary from a hetzner cloud Ubuntu 20.04.6 VM:

root@ubuntu-2gb-hil-1:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"

Which seems to be the same version as the runners:

 + cat /etc/os-release
  NAME="Ubuntu"
  VERSION="20.04.6 LTS (Focal Fossa)"

Still getting exec format error though.

[jseparovic]

I managed to get the workaround working on Terraform Cloud. From my previous message, I think I'd messed up the curl command which I copied from the Terraform docs. When I tested it locally, it created a 0 byte file. Anyway I swapped to using wget and it works now.

If anyone is interested, this is what I added to get it working:

resource "null_resource" "always_run" {
  triggers = {
    timestamp = timestamp()
  }
}

resource "terraform_data" "cluster" {
  lifecycle {
    replace_triggered_by = [
      null_resource.always_run
    ]
  }

  # Example PATH in a runner
  # /home/tfc-agent/.tfc-agent/component/terraform/runs/run-bDb4JfFca6LSu4Jv/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/local/bin
  provisioner "local-exec" {
    command = <<EOH
set -x
echo PATH=$PATH
cat /etc/os-release
uname -a
pwd
mkdir -p ../.local/bin
wget https://github.com/jseparovic/ubuntu2004/raw/main/mkisofs -O ../.local/bin/mkisofs
chmod 0755 ../.local/bin/mkisofs
mkisofs -version
set +x
EOH
    }
}

I ended up building genisoimage (aka: mkisofs) from source as per https://github.com/jseparovic/ubuntu2004

A note on the PATH, it seems that the runner already has a PATH entry for ./.local/bin so all you have to do is create this folder and put any binaries in there for them to work as is.

[scabala]

Related #895

As mentioned in #935, probably best approach is to move to vfat for cloud-init

[abakanovskii]

For anyone else stumbling upon this: apt install genisoimage (you might not have to build it from source as in example mentioned before)