Schema validation fails on jinja template

[ubuntu-server-builder]

This bug was originally filed in Launchpad as LP: #1881925

Launchpad details

affected_projects = []
assignee = None
assignee_name = None
date_closed = None
date_created = 2020-06-03T14:57:36.281765+00:00
date_fix_committed = None
date_fix_released = None
id = 1881925
importance = medium
is_complete = False
lp_url = https://bugs.launchpad.net/cloud-init/+bug/1881925
milestone = None
owner = falcojr
owner_name = James Falcon
private = False
status = in_progress
submitter = falcojr
submitter_name = James Falcon
tags = []
duplicates = []

Launchpad user James Falcon(falcojr) wrote on 2020-06-03T14:57:36.281765+00:00

$ cat > test.yaml << EOF

template: jinja

cloud-config

runcmd:

• echo 'hostname: {{ ds.meta_data.public_hostname }}'

EOF $ cloud-init devel schema -c test.yaml Cloud config schema errors: format-l1.c1: File test.yaml needs to begin with "#cloud-config"

[ubuntu-server-builder]

Launchpad user Chad Smith(chad.smith) wrote on 2020-06-03T16:59:32.192720+00:00

good bug, needs wiring into cloudinit/handlers/jinja_template:render_jinja_payload_from_file I believe.

[ubuntu-server-builder]

Launchpad user Dan Watkins(oddbloke) wrote on 2020-06-03T21:08:32.855860+00:00

I think it's a little more complex than just wiring up. Schema validation doesn't only happen on systems which are the target for the cloud-config being validated (e.g. I could want to validate a schema which uses Azure-specific substitutions on my local (obviously-non-Azure) machine). And, more strongly than that, schema validation doesn't even only happen on systems where cloud-init has run or is installed (my dev machine doesn't have cloud-init installed). So we cannot assume that there will be any cloud-init data to substitute in and even if there is, we cannot assume that it is appropriate for use in the given cloud-config template.

As far as our schema is concerned, the specific values aren't important, it's just the types of the values that matter. So we can solve this by producing fake values for the variables in the template somehow. (We'll need to think about types a bit, but this isn't unachievable by any means.)

But we still have a problem: templates can have control flow. So a Jinja2 template can actually produce structurally different cloud-configs, depending on the inputs to it. And, of course, the structure of cloud-config is what the schema validates. I don't really have a great idea on how to handle this problem. That said, I think this applies to a small proportion of templates, so it's still absolutely worth enabling validation of the simple and common case on its own.

[ubuntu-server-builder]

Launchpad user Chad Smith(chad.smith) wrote on 2023-04-19T18:28:03.335967+00:00

Pull request proposed upstream to render ## template: jinja content and validate schema of resulting #cloud-config user-data.

https://github.com/canonical/cloud-init/pull/2132