custom_data doesn't appear to be running.

[charlie-ava]

I'm kind of new to azure so I'm probably missing something but have been looking at it for 2 days and hoping someone has an easy answer. We are trying to create a single azure vm and the custom_data does not appear to be running. We use packer and ansible to build out a base image then try booting a virtual machine using that image but I can not see in waagent log or really anything where it's running the custom data. Our custom_data script is just a simple bash script that touches some files and echos a variable so we can validate it. Also it does not appear to change the hostname from the packer name (I'm assuming packer some how is a culprit).:q But I never see anything. Here is my code so you can see what I'm doing.

resource "azurerm_linux_virtual_machine" "this" { name = var.name computer_name = var.name location = var.location resource_group_name = azurerm_resource_group.this.name network_interface_ids = [azurerm_network_interface.this.id] size = var.instance_type admin_username = var.username admin_password = random_password.password.result source_image_id = data.azurerm_image.ubuntu.id disable_password_authentication = false encryption_at_host_enabled = true custom_data = base64encode(data.template_file.cloud-init.rendered)

os_disk { caching = "ReadWrite" storage_account_type = "Standard_LRS" disk_size_gb = var.volume_size } }

data "template_file" "cloud-init" { template = file("${path.module}/../scripts/${var.user_data}") vars = { component = var.component } }

script looks like this:

!/bin/bash

touch /tmp/file echo $component /tmp/file

[Lucretius]

You don't need to base64encode the custom_data, Terraform does that for you automatically. Your custom data is being base64encoded twice, and thus is not understood by the VM.

Change custom_data = base64encode(data.template_file.cloud-init.rendered) to custom_data = data.template_file.cloud-init.rendered

[charlie-ava]

So I tried this, but it fails with the error, is it possible I need to move the base64 encoding into the file:

Error: expected "custom_data" to be a base64 string, got #!/bin/bash touch /tmp/file echo $component > /tmp/file

[charlie-ava]

So I decided to test this out and did a "cat user-data.sh | base64 > user-data.64" However, I still get a base64 error:

Error: creating Linux Virtual Machine "REDACTED" (Resource Group "REDACTED"): compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidParameter" Message="Custom data in OSProfile must be in Base64 encoding and with a maximum length of 87380 characters." Target="customData"

[Lucretius]

I think there is a maximum length to custom data that can be passed in, enforced by Azure. Perhaps your script is too long?
The error message does not say if it is the base64 encoding or the length causing the problem - just that your script must match those length and encoding requirements. I normally use heredocs and inline my custom data and this works just fine. Something like:

custom_data = <<EOF
#!/bin/bash
echo "hello"
EOF

[charlie-ava]

It's still not happy without base64

Error: expected "custom_data" to be a base64 string, got #!/bin/bash touch /tmp/file echo "Hello world" > /tmp/file

[jackofallops]

Hi @charlie-ava - I can confirm that the string supplied to the provider should be a base64 encoded string (or use the inbuilt base64encode function). e.g.

custom_data = base64encode("hostname")

My recommendation would be to see the cloud-init logs on the deployed server to investigate what's happening at server-side. For example, the image you're running may not have that shell available? (albeit unlikely that's the case with Ubuntu) (ref: https://cloudinit.readthedocs.io/en/latest/topics/faq.html#where-are-the-logs)

[Lucretius]

Yeah I took a second look at my code and realized I do indeed wrap my heredocs in the base64encode function, try what @jackofallops had recommended and it should work.

[charlie-ava]

So custom_data with base64encode works fine as long as I'm not using packer. There is something that it is doing that makes custom_data not work. Perhaps I might need to bug the hashicorp guys to find out what is not getting cleaned up as the waagent deprovision in their docs doesn't seem to do it.

[Lucretius]

Glad the custom_data is working now. As for the Packer issue, are you referring to trying to invoke Packer in your Terraformed VM? Just a guess, but I have had issues with Packer and custom data with Linux in the past (namely trying to install Ubuntu packages via Packer) that I resolved by adding the following line at the beginning of my Packer inline scripts:

while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init...'; sleep 1; done

According to the docs, "The problem arises when cloud-init has not finished fully running on the source AMI by the time that packer starts any provisioning steps."

See this link for more info. If that fails then yeah, I'd go open an issue on the Packer repo and see if they can provide more help.

[darrens280]

Hi.

For me, it was because the PowerShell script I was injecting into the guest VM, using the custom_data method under OS_profile section, was too many characters.

Error received: Error: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidParameter" Message="Custom data in OSProfile must be in Base64 encoding and with a maximum length of 87380 characters." Target="customData"

As per Terraform docs, regarding custom_data, "the maximum length of the binary array is 65535 bytes"

According to my VSCode my .PS1 file was 65674 characters long. I trimmed down some comment sections as a test to 65372 characters, but Terraform still failed with same message. So I trimmed some more, to get it 64439. Now Terraform successfully applies, and injects the script without error.

Example here (Just highlight all characters in script with Ctrl+A to see the total character count):

Hope this helps someone out there. Cheers Darren

[jwefers]

I just also tripped over the issue that custom_data does NOT actually get auto-converted to base64 - could you guys please fix the documentation? Because it says it would:

https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/virtual_machine_scale_set#custom_data https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/linux_virtual_machine#custom_data and prob. same for Windows. Thanks :)

[chunkingz]

I agree with @jwefers I have just experienced the same thing as I am somewhat new to terraform from ansible.

I had to make use of the base64encode method like so

custom_data = base64encode(file("azure-user-data.sh"))

as custom_data = file("azure-user-data.sh") will always give you this error:

Error: expected "custom_data" to be a base64 string, got #! /bin/bash
...

it would be best if they update the documentation to cover this aspect, or update the code base to take care of this auto-encoding to base64

[mjrlgue]

same here, shell script don't get executed

[mjrlgue]

It's me again, I found out (after searching a lot of blogs and docs and trying many code snippets) there is a tool called cloud-init, it's intended to run some code when the vm is provisioned. Below the code I tested on Azure:

main.tf:

data "template_file" "script" {
  template = file("scripts/update-vm.yaml")
}

data "template_cloudinit_config" "config" {
  gzip          = true
  base64_encode = true

  # Main cloud-config configuration file.
  part {
    content_type = "text/cloud-config"
    content      = data.template_file.script.rendered
  }
}

resource "azurerm_linux_virtual_machine" "my-vm" {
  name                = "my-vm"
  resource_group_name = data.azurerm_resource_group.my-rg.name
  location            = "France Central"
  size                = "Standard_B2ms"
  admin_username      = "adminuser"
  computer_name       = "node1" # set the host
  network_interface_ids = [
    azurerm_network_interface.my-nic-nic.id,
  ]

  custom_data = data.template_cloudinit_config.config.rendered

  admin_ssh_key {
    username   = "adminuser"
    public_key = file("~/.ssh/myKey.pub")
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Standard_LRS"
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "18.04-LTS"
    version   = "latest"
  }
}

and

update-vm.yaml

#cloud-config
users:
  - default
  - name: myUser  # here I created a new user and added it to sudo group
    groups: sudo
    shell: /bin/bash
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    ssh-authorized-keys:
      - ssh-rsa AAAAB3... # paste your public key here

packages: ['httpie'] # to install a package, you pass many packages in the list

# bootcmd:
  # - echo 192.168.1.130 us.archive.ubuntu.com >> /etc/hosts

runcmd:
  - sudo su myUser
  - cd /home/myUser
  - wget https://some-url.tar.gz
  - tar -xf *.tar.gz
  - rm -f *.tar.gz

When running the above code using terraform and the VM is started, you'll see the file downloaded on the location you specified in runcmd, the bootcmd let you perform some action when the VM is starting (uncomment it to test). Examples of cloud-init can be found here.

Hope it helps someone!

[franroad]

Hello there, I have been facing the same issue, finally I found the following vid[1] where the solution is described. In a nutshell: 1- Create a file with the script it self eg: "ansible.sh" (installing ansible):

#!/bin/bash
sudo apt-add-repository ppa:ansible/ansible
sudo apt update
sudo apt install -y ansible

2- Add the following line inside the resource (VM) block:

custom_data = filebase64("ansible.sh") # specify the directory in case it is not in the same directory

After accessing the vm and running "ansible --version" , ansible was properly installed

[1].https://www.youtube.com/watch?v=ajD9VpcKRUE