jinja templates are not honored in cloud-init config

[dominikborkowski]

We'd like to use multipass to speed up prototyping cloud-init configurations used with AWS on our local systems. Unfortunately, more complex configurations leveraging jinja templates are currently not possible, as multipass strips out relevant headers from the user-data, along with anything that may be commented out and containing valid jinja commands.

For example, this input cloud-init config:

## template: jinja
#cloud-config
#
# cloud-init config : create packer instance
#
# Jinja variables:
# {% set PACKER_VER='1.5.1' %}

package_update: true
package_upgrade: true

Should result in the following /var/lib/cloud/instance/user-data.txt (as tested on EC2):

## template: jinja
#cloud-config
#
# cloud-init config : create packer instance
#
# Jinja variables:
# {% set PACKER_VER='1.5.1' %}

package_update: true
package_upgrade: true

Instead, we lose the required header and variable setting:

#cloud-config
package_update: true
package_upgrade: true

This severely limits our options of using multipass for the goal of being a local stage area for cloud-init, and we'd love to avoid having to use external scripts or other workarounds.

Thanks!

[Saviq]

Thanks for the report @dominikborkowski, what is happening is that the comments are lost when parsing and emitting again with yaml-cpp (which does, somewhat, make sense).

We're talking about the broader picture of cloud-init support in Multipass, we'll definitely take this into account.

[jbrisbin]

What was ever decided on this? I'd like to use the cloud-init functionality but I need variable substitution and/or templates to keep things like secrets outside the clout-init.yaml that I'm using with multipass. Also for setting up a Kubernetes cluster I need to parameterize things so I can create multiple, numbered nodes. It doesn't seem like there's any processing done on the YAML that's loaded, so I have to assume there's no variable substitution being done, nor templating in the C++ code, is that correct?

[Saviq]

Hi @jbrisbin, we're pushing the YAML as-is, so any templating needs to happen outside of Multipass. Would a cat template.yaml | <template-tool> | multipass launch --cloud-init - work for you?

[jbrisbin]

I can template the YAML prior to sending it to multipass (gomplate, jinja, etc...) but that requires an external dependency/tool and is less portable than by including support for templating inside the YAML parsing of multipass IMO.

[Saviq]

@jbrisbin we do plan to expose more cloud-init features indeed, with time. Sorry we can't be of help right now.

[joes]

I tried using instance metadata in runcmd as described here:

https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html#using-instance-data

This too did not work and I am guessing the problem is the one described in this issue.

Workaround is to write a separate template (with write_files) and make runcmd render the template with jinja and execute the result:

runcmd:
 - cloud-init query --format="$( cat /var/cloud-init/runcmd.sh.template )" > /var/cloud-init/runcmd.sh
 - chmod 600 /var/cloud-init/runcmd.sh
 - echo "runcmd is:"
 - cat /var/cloud-init/runcmd.sh
 - /usr/bin/env bash -f /var/cloud-init/runcmd.sh

[jafudi]

The exact same issue is still present in version 1.8.1+mac. Any updates on this? I am somewhat disappointed because cloud-init and multipass are both invented by Canonical and should be super compatible. Please try to fix this! Many thanks.

[gz-95]

Without jinja it's hard to install packages that depend on instance data like:

## template: jinja
#cloud-config

packages:
  - linux-modules-extra-{{ v1.kernel_release }}