Open chrisoverzero opened 4 months ago
Thanks for raising this issue. We're open to accepting this request and can chat here about implementation details, that way other team members can find this issue and reply in case they had more context or ideas.
Regarding specification changes (point no.1), would the change to Function.ImageUri
to potentially expect a tarball be the only change you imagine would be required to support this?
Regarding point no.4, this should be possible. I've never worked with building images from archives, but we build a Docker image from a tarball that contains things like the Dockerfile and required binaries, as part of the local emulation behaviour: https://github.com/aws/aws-sam-cli/blob/a18a7292227d28a0f497ab44177ccf9a41174ced/samcli/local/docker/lambda_image.py#L345-L355 The documentation for the build API can be found here: https://docker-py.readthedocs.io/en/stable/images.html#docker.models.images.ImageCollection.build
If this was the same kind of thing you had in mind, then it should be possible to do the same thing for the building context.
There also looks to be a loading API if the archive was exported: https://docker-py.readthedocs.io/en/stable/images.html#docker.models.images.ImageCollection.load
Regarding specification changes (point no.1), would the change to Function.ImageUri to potentially expect a tarball be the only change you imagine would be required to support this?
I worry that this is a laser-guided question and that you, who knows this code base better than I do, already know about something I've missed! 😅 But I don't believe so. What I envision is that this is a preprocessing step, effectively. Once the archive is docker image load
ed into an image, the regular image code path can take over. I don't think the server-side processing would need to change at all. The documentation would need to mention the local path and probably have a note similar to CodeUri
:
[!NOTE]
If you provide a local file path, use the AWS SAM CLI to load and push the local archive as an image at deployment. To learn more, see Using the AWS SAM CLI to upload local files at deployment.
But I'm sure I'll discover whatever I'm missing when I dive into the code.
There also looks to be a loading API if the archive was exported:
That's exactly what I was looking for – thanks for getting directly there. There doesn't seem to be an equivalent to docker image import
, but I've also realized that the tag used is arbitrary, since it's passed through mechanically. So when the archive is loaded into an image via that method, we can choose the first tag arbitrarily. Easy-peasy, I hope.
I don't currently have any issues with using the ImageUri to serve up a local archive, so we can explore that path to implement this.
We also have the Metadata
section of an AWS::Serverless::Function
, this could also be considered in addition/as an alternative to ImageUri.
The Metadata
section would contain SAM CLI specific behaviour (not affecting Cloudformation) and could be a candidate to specify an archive file. The changes to the Metadata
might be breaking depending on how we go about it, but just another idea.
Docs: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-build.html Example template: https://github.com/aws/aws-sam-cli/blob/c4a4cdbe48edd95374624661a34350f12ae09b93/tests/integration/testdata/start_api/image_package_type/template.yaml
For the record, I have my proof-of-concept running as of this morning (EDT) using ImageUri
. I can sam build
then sam deploy
a real application right up to our test account. ~I haven't started on sam package
yet.~ I feel strongly that sam package
should work directly from path-flavored-ImageUri
without requiring sam build
beforehand, but if that's violating some larger design principle, please do let me know.
I'm confident in meeting the team's standards for documentation, but it's going to take me a bit to figure out how best to add this to automated testing. I'm unlikely to open an MR until I think it's ready to go unless you tell me otherwise.
Later: I have sam package
working. The whole thing needs to be nicer, but I've got it goin'. Using the first tag arbitrarily isn't going to work for sam package
because it can pick up the ECR-repo'd tag from a previous invocation of sam package
, so I'll figure something out there.
I don't yet consider that PR complete because I don't know what documentation the required step "Write documentation" refers to. Is it about documenting methods/functions? Or is it about updating documentation in the SAM Spec? Or a third thing?
Describe your idea/feature/enhancement
I wish SAM CLI would accept a local path to an image archive (the kind of thing which can be loaded by
docker image load
) for the value ofImageUri
.Proposal
For an
ImageUri
containing a local path to an image archive, the commandsam build
would perform the equivalent ofdocker image load
(in the same way that it will currently perform the equivalent ofdocker image build
) and write the image name and tag into the built template as the new value ofImageUri
.The command
sam package
would do the same thing, but additionally perform the equivalent of thedocker image push
and write the address of the image in the remote repository into the output template as the new value ofImageUri
.In general, it would make
ImageUri
work a little more likeCodeUri
.Things to consider:
ImageUri
property would have to be documented to accept this kind of value.docker image import
could be performed instead, assigning a single canonical, generated name to the image in the same way assam build
does.docker
command-line tool or the docker service available. I know of other development teams who produce file archives of Docker images for scanning purposes. Lacking either end of Docker simplifies those steps in the CI process, so we only need to associate the docker-in-docker service with the job in which we performsam
commands for the final deployment. Also, it's much easier in many ways to manage CI artifacts which are normal files.sam build
does its Docker work without thedocker
CLI tool available by usingdocker+http://
URLs somehow. I presume it's issuing commands to the Docker service over a more direct interface, and I see this capability in the Docker API, so it feels possible.ImageUri
like this is undesirable, I could imagine a property namedImagePath
orImageArchive
or something similar to serve the same purpose.Additional Details
I would be willing to take on this work if this request is accepted. I've contributed to serverless-application-model and aws-sam-build-images before, so I probably wouldn't be completely in over my head.