Open dustindortch opened 3 years ago
Hey there, thanks for reaching out!! If I understood correctly your intentions, you can achieve that using dynamic source block within your build. I rewrote your template with what I'm suggesting.
variables {
images = {
windows_server_2019 = {
communicator = "winrm"
publisher = "MicrosoftWindowsServer"
offer = "WindowsServer"
sku = "2019-Datacenter"
type = "Windows"
}
sql_server_2019 = {
communicator = "winrm"
publisher = "microsoftsqlserver"
offer = "sql2019-ws2019"
sku = "standard"
type = "Windows"
}
}
}
source "azure-arm" "golden-image" {
# Common elements
tenant_id = var.TENANT_ID
subscription_id = var.SUBSCRIPTION_ID
client_id = var.CLIENT_ID
client_secret = var.client_secret
managed_image_resource_group_name = var.RESOURCE_GROUP_NAME
managed_image_name = "${var.PREFIX}${each.key}_${formatdate("YYYY-MM-DD_hh-mm-ss", timestamp())}"
managed_image_storage_account_type = "Premium_LRS"
location = var.LOCATION
vm_size = var.VM_SIZE
winrm_use_ssl = true
winrm_insecure = true
winrm_timeout = "5m"
winrm_username = "packer"
}
build {
dynamic "source" {
for_each = var.images
labels = ["azure-arm.golden-image"]
content {
name = source.key
image_publisher = source.value.publisher
image_offer = source.value.offer
image_sku = source.value.sku
os_type = source.value.type
communicator = source.value.communicator
}
}
# Build stuff
}
Let me know if that works for you. 😄
What about a case where I have multiple platforms and I want to loop through a source for one platform and then a source for another platform? I think the for_each logic would be an improvement.
In this case — each builder (or platform) — has a very specific set of fields, behaviours and settings one can set, and I think it is better to define them in their own source block. So if you wanted to run the same provisioning steps on a different source, you'd have to do another dynamic "source"
call, for example:
dynamic "source" {
for_each = var.azure_images
labels = ["azure-arm.golden-image"]
content {
name = source.key
image_publisher = source.value.publisher
// ...
}
dynamic "source" {
for_each = var.gcp_images
labels = ["googlecompute.golden-image"]
content {
name = source.key
account_file = source.value.account_file
// ...
}
+1 to this request.
For a bit more context, imagine you have a matrix of sources and builds. For example, you have 3 different AWS accounts and 3 different builds and you want each build to run in each AWS account (total of 9 images created). Assume your provisioner is going to take the account & build as input and configure each of the 9 images differently so you can't just build a single image and share it between accounts.
Only allowing a dynamic source inside of the builder means anything that has to adjust between accounts (ex. security groups) has to be duplicated across all of the builds. This is currently leading us to lots of copy/paste code. To get around this, we're instead generating our hcl2 with other scripts which is also suboptimal.
Consider this working code as an example:
source "null" "first-example" {
communicator = "none"
}
build {
name = "roles"
dynamic source {
for_each = ["consul", "nomad", "vault"]
labels = ["null.first-example"]
content {
name = source.value
}
}
provisioner "shell-local" {
inline = ["echo hello ${source.name} and ${source.type}"]
}
}
build {
name = "roles"
dynamic source {
for_each = ["consul", "nomad", "vault"]
labels = ["null.first-example"]
content {
name = source.value
}
}
provisioner "shell-local" {
inline = ["echo goodbye ${source.name} and ${source.type}"]
}
}
If dynamic top level sources were possible, I think this would be much easier to understand and to maintain.
dynamic "source" {
for_each = ["consul", "nomad", "vault"]
content {
type = "null"
name = source.value
communicator = "none"
}
}
build {
name = "roles"
sources = [
"null.consul",
"null.nomad",
"null.vault",
]
provisioner "shell-local" {
inline = ["echo hello ${source.name} and ${source.type}"]
}
}
build {
name = "roles"
sources = [
"null.consul",
"null.nomad",
"null.vault",
]
provisioner "shell-local" {
inline = ["echo goodbye ${source.name} and ${source.type}"]
}
}
This can also apply for provisioners.
So if you wanted to run the same provisioning steps on a different source, you'd have to do another
dynamic "source"
call, for example:
Thank you so much for this example. I spent hours trying to get the right for_each and dynamic source blocks to cross-build a container image for multiple architectures. This isn't nearly as intuitive as it should be, and it seems Packer is missing the toset
function also? This example will cross-build docker containers for a source container image across x86_64, aarch64, and riscv64 all in a single block. Super helpful.
build {
dynamic "source" {
for_each = {arch="x86_64",arch="aarch64",arch="riscv64"}
labels = ["source.docker.base_ubuntu"]
content {
name = source.value
platform = "linux/${source.value}"
}
}
}
I would think this is a very common need. Some examples in the documentation would be very helpful.
@jboero Are you sure that "set" syntax works?
for_each = {arch="x86_64",arch="aarch64",arch="riscv64"}
I had to do
for_each = ["x86_64", "aarch64", "riscv64"]
When working with a provider, it would great to be able to use a for_each loop in the source block for a golden image pipeline where you have different source images from the same provider that you want to adhere to your golden image standards.
For instance, with Azure, I'd like a golden image for Windows Server 2019, but I need a regular server and a SQL Server instance:
so, for the source:
This would be made with a variable definition, as such:
Then, the variables could be build out:
Finally, in the build:
Then the various golden images could be built in parallel without having to define a separate source blockfor each one. Alternatively, you could have multiple source blocks if you have multiple platforms, but you could have multiple source images for each platform.