Feature request: AWS::Serverless::Function.ImageUri Should Also Accept a Local File Path

[chrisoverzero]

Describe your idea/feature/enhancement

I wish SAM CLI would accept a local path to an image archive (the kind of thing which can be loaded by docker image load) for the value of ImageUri.

Proposal

For anImageUri containing a local path to an image archive, the command sam build would perform the equivalent of docker image load (in the same way that it will currently perform the equivalent of docker image build) and write the image name and tag into the built template as the new value of ImageUri.

The command sam package would do the same thing, but additionally perform the equivalent of the docker image push and write the address of the image in the remote repository into the output template as the new value of ImageUri.

In general, it would make ImageUri work a little more like CodeUri.

Things to consider:

1. Will this require any updates to the SAM Spec. Yes, the ImageUri property would have to be documented to accept this kind of value.
2. What if the archive is associated with multiple tags? If we have the taste for it, that could be an error. Alternatively, an action equivalent to docker image import could be performed instead, assigning a single canonical, generated name to the image in the same way as sam build does.
3. Why would anyone want this? W-Well, I want this because my build system is capable of creating Docker images as file archives without the docker command-line tool or the docker service available. I know of other development teams who produce file archives of Docker images for scanning purposes. Lacking either end of Docker simplifies those steps in the CI process, so we only need to associate the docker-in-docker service with the job in which we perform sam commands for the final deployment. Also, it's much easier in many ways to manage CI artifacts which are normal files.
4. Is this a thing that SAM CLI can possibly do? I admit I don't know for certain. sam build does its Docker work without the docker CLI tool available by using docker+http:// URLs somehow. I presume it's issuing commands to the Docker service over a more direct interface, and I see this capability in the Docker API, so it feels possible.
5. Is there an alternative design? If overloading ImageUri like this is undesirable, I could imagine a property named ImagePath or ImageArchive or something similar to serve the same purpose.

Additional Details

I would be willing to take on this work if this request is accepted. I've contributed to serverless-application-model and aws-sam-build-images before, so I probably wouldn't be completely in over my head.

[lucashuy]

Thanks for raising this issue. We're open to accepting this request and can chat here about implementation details, that way other team members can find this issue and reply in case they had more context or ideas.

Regarding specification changes (point no.1), would the change to Function.ImageUri to potentially expect a tarball be the only change you imagine would be required to support this?

Regarding point no.4, this should be possible. I've never worked with building images from archives, but we build a Docker image from a tarball that contains things like the Dockerfile and required binaries, as part of the local emulation behaviour: https://github.com/aws/aws-sam-cli/blob/a18a7292227d28a0f497ab44177ccf9a41174ced/samcli/local/docker/lambda_image.py#L345-L355 The documentation for the build API can be found here: https://docker-py.readthedocs.io/en/stable/images.html#docker.models.images.ImageCollection.build

If this was the same kind of thing you had in mind, then it should be possible to do the same thing for the building context.

There also looks to be a loading API if the archive was exported: https://docker-py.readthedocs.io/en/stable/images.html#docker.models.images.ImageCollection.load

[chrisoverzero]

Regarding specification changes (point no.1), would the change to Function.ImageUri to potentially expect a tarball be the only change you imagine would be required to support this?

I worry that this is a laser-guided question and that you, who knows this code base better than I do, already know about something I've missed! 😅 But I don't believe so. What I envision is that this is a preprocessing step, effectively. Once the archive is docker image loaded into an image, the regular image code path can take over. I don't think the server-side processing would need to change at all. The documentation would need to mention the local path and probably have a note similar to CodeUri:

[!NOTE]
If you provide a local file path, use the AWS SAM CLI to load and push the local archive as an image at deployment. To learn more, see Using the AWS SAM CLI to upload local files at deployment.

But I'm sure I'll discover whatever I'm missing when I dive into the code.

There also looks to be a loading API if the archive was exported:

That's exactly what I was looking for – thanks for getting directly there. There doesn't seem to be an equivalent to docker image import, but I've also realized that the tag used is arbitrary, since it's passed through mechanically. So when the archive is loaded into an image via that method, we can choose the first tag arbitrarily. Easy-peasy, I hope.

[lucashuy]

I don't currently have any issues with using the ImageUri to serve up a local archive, so we can explore that path to implement this.

We also have the Metadata section of an AWS::Serverless::Function, this could also be considered in addition/as an alternative to ImageUri.

The Metadata section would contain SAM CLI specific behaviour (not affecting Cloudformation) and could be a candidate to specify an archive file. The changes to the Metadata might be breaking depending on how we go about it, but just another idea.

Docs: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-build.html Example template: https://github.com/aws/aws-sam-cli/blob/c4a4cdbe48edd95374624661a34350f12ae09b93/tests/integration/testdata/start_api/image_package_type/template.yaml

[chrisoverzero]

For the record, I have my proof-of-concept running as of this morning (EDT) using ImageUri. I can sam build then sam deploy a real application right up to our test account. ~I haven't started on sam package yet.~ I feel strongly that sam package should work directly from path-flavored-ImageUri without requiring sam build beforehand, but if that's violating some larger design principle, please do let me know.

I'm confident in meeting the team's standards for documentation, but it's going to take me a bit to figure out how best to add this to automated testing. I'm unlikely to open an MR until I think it's ready to go unless you tell me otherwise.

Later: I have sam package working. The whole thing needs to be nicer, but I've got it goin'. Using the first tag arbitrarily isn't going to work for sam package because it can pick up the ECR-repo'd tag from a previous invocation of sam package, so I'll figure something out there.

[chrisoverzero]

I don't yet consider that PR complete because I don't know what documentation the required step "Write documentation" refers to. Is it about documenting methods/functions? Or is it about updating documentation in the SAM Spec? Or a third thing?