getmoto / moto

A library that allows you to easily mock out tests based on AWS infrastructure.
http://docs.getmoto.org/en/latest/
Apache License 2.0
7.62k stars 2.04k forks source link

Mocking lambda using Python 3.12 Runtime fails: `manifest for lambci/lambda:python3.12 not found` #8236

Open chris48s opened 1 day ago

chris48s commented 1 day ago

I'm trying to write tests for a lambda function using the python 3.12 runtime.

This is failing with:

error running docker: 404 Client Error for http+docker://localhost/v1.43/images/create?tag=python3.12&fromImage=lambci%2Flambda: Not Found ("manifest for lambci/lambda:python3.12 not found: manifest unknown: manifest unknown")

Looking over the code here:

https://github.com/getmoto/moto/blob/77fbf518edbd78a427823380cc530bda079799c1/moto/awslambda/models.py#L1016-L1028

it looks like moto first tries to find an appropriate docker image for the target runtime in https://hub.docker.com/r/mlupin/docker-lambda/tags and then when it can't find a python 3.12 image there it falls back to https://hub.docker.com/r/lambci/lambda/tags

and then fails with that error.

I accept this isn't fundamentally a bug in moto and you can boil it down to "someone needs to go work on https://github.com/mLupine/docker-lambda/issues/26 upstream"

The thing I find surprising about this is that moto relies on third-party docker images and not the lambda images that AWS publish themselves: https://gallery.ecr.aws/lambda/python

Looking back over some old issues/PRs e.g:

I cant really see any discussion of this, but I assume there is a reason. Anyone able to shed any light on this?

It also doesn't seem possible to tell moto to use (for example) public.ecr.aws/lambda/python:3.12 with the MOTO_DOCKER_LAMBDA_IMAGE var due to the difference in naming conventions.

rafcio19 commented 15 hours ago

@bblommers if you're ok with this change then I can implement images from https://gallery.ecr.aws/lambda/, would make it the first repo to check.

bblommers commented 8 hours ago

Hi @chris48s!

I assume there is a reason

When we started using the lambCI images (I think back in 2017), the official AWS images weren't available yet. So we didn't really have a choice. :slightly_smiling_face:

Supporting the official repositories makes sense, although I'd be a little worried about making it the default option. Technically it should be interchangeably, but I don't have enough insight in the makeup of both Docker images to be confident that we can do that in a patch release and expect everything to work as before.

How about we just support MOTO_DOCKER_LAMBDA_IMAGE=public.ecr.aws/lambda/python:3.12? That's a one-line change, if I'm not mistaken:

- image_ref = f"{image_repo}:{self.run_time}"
+ image_ref = image_repo if ":" in image_repo else f"{image_repo}:{self.run_time}"