Build custom packer image for GCP

[eric-czech]

I'm trying to follow the instructions in https://cloudprovider.dask.org/en/latest/packer.html#ec2cluster-with-cloud-init for GCP, but I'm stuck on how to get the packer configuration to recognize the cloud init configuration. For amazon, that works by doing this:

{
    "builders": [
        {
            ...
            "type": "amazon-ebs",
            "user_data_file": "/path/to/cloud-init-config.yaml"
        }
    ],
    "provisioners": ...
}

The closest equivalent (according to https://cloud.google.com/container-optimized-os/docs/how-to/create-configure-instance) for GCP appears to be:

{
    "builders": [
        {
            ...
            "type": "googlecompute",
            "metadata": {
                "user-data": "/path/to/cloud-init-config.yaml"
            }
        }
    ],
    "provisioners": ...
}

This isn't working for me though. The cloud-init-config.yaml file is being ignored and I'm not sure why.

Do you guys know how to make this work? Have you tried this yet @quasiben?

[quasiben]

Thanks @eric-czech for the issue. @jacobtomlinson has a gist going over this exact setup: https://gist.github.com/jacobtomlinson/15404d5b032a9f91c9473d1a91e94c0a

This should be listed in the docs at some point

[dbalabka]

I've implemented a custom image building with Packer, as described in the gist, right before the cluster started. It works great and minimizes startup time. I want to let you know that you must stick with one image name to force the parcel to reuse the image instead of creating it each time.

Please comment here: https://gist.github.com/jacobtomlinson/15404d5b032a9f91c9473d1a91e94c0a?permalink_comment_id=5191024#gistcomment-5191024