[Missing Documentation] - Clarify which manylinux platform must be used with AL2023 for Python Lambda functions

[PierreKiwi]

Hello!

We are working a lot with Python and we started walking the path towards Python 3.12 for our Lambda functions.

We are wondering if the following article - https://repost.aws/knowledge-center/lambda-python-package-compatible - is still relevant for AL3 ? More specifically around the value of --platform.

We haven't found in the documentation what's version of Fedora AL3 is based on, so we are unsure if we should keep using manylinux2014 or manylinux_2_28 when packaging our Python 3.12 Lambda function (https://github.com/pypa/manylinux#manylinux).

[stewartsmith]

See https://docs.aws.amazon.com/linux/al2023/ug/relationship-to-fedora.html for discussion on the AL2023 relationship to Fedora.

As per https://docs.aws.amazon.com/linux/al2023/ug/core-glibc.html there's glibc 2.34 in AL2023 (see also https://docs.aws.amazon.com/linux/al2023/release-notes/all-packages-AL2023.3.html for a full package list) - so that can be used for the manylinux_x_y formula for glibc version.

[PierreKiwi]

Great, thanks a lot @stewartsmith !

Actually I also realised that I can specify multiple values --platform so the packaging of my functions seems fine now with --platform manulinux2014 and --platform manylinux_2_28. Will formally test running the functions early next year.

[stewartsmith]

Do you think it would be of assistance if we put something relating to this over on https://docs.aws.amazon.com/linux/al2023/ug/python.html ?

[PierreKiwi]

Do you think it would be of assistance if we put something relating to this over on https://docs.aws.amazon.com/linux/al2023/ug/python.html ?

Apologies for the delay @stewartsmith. That could be helpful indeed, but I believe it will be more helpful if the Lambda team updates their doc as well (especially this section => https://docs.aws.amazon.com/lambda/latest/dg/python-package.html#python-package-native-libraries)

[ascopes]

@stewartsmith As per https://docs.aws.amazon.com/linux/al2023/ug/core-glibc.html there's glibc 2.34 in AL2023 (see also https://docs.aws.amazon.com/linux/al2023/release-notes/all-packages-AL2023.3.html for a full package list) - so that can be used for the manylinux_x_y formula for glibc version.

There doesn't appear to be a manylinux container image released by pypa for manylinux_2_34, and many packages like cryptography do not provide a manylinux_2_34 distribution, so what is the advice regarding this?

https://pypi.org/project/cryptography/41.0.7/#files

https://quay.io/organization/pypa

Libraries like Cryptography using the old wheels just fail due to issues with glibc, even with cffi installed.

<module 'cffi' from '/opt/python/cffi/__init__.py'> 1.16.0
...
<module 'cryptography' from '/opt/python/cryptography/__init__.py'> 41.0.7
...
ModuleNotFoundError: No module named '_cffi_backend'
thread '<unnamed>' panicked at /github/home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/err/mod.rs:790:5:
Python API call failed
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
File "/var/runtime/bootstrap.py", line 63, in <module>
main()
File "/var/runtime/bootstrap.py", line 60, in main
awslambdaricmain.main([os.environ["LAMBDA_TASK_ROOT"], os.environ["_HANDLER"]])
File "/var/lang/lib/python3.12/site-packages/awslambdaric/__main__.py", line 21, in main
bootstrap.run(app_root, handler, lambda_runtime_api_addr)
File "/var/lang/lib/python3.12/site-packages/awslambdaric/bootstrap.py", line 494, in run
handle_event_request(
File "/var/lang/lib/python3.12/site-packages/awslambdaric/bootstrap.py", line 186, in handle_event_request
response = request_handler(event, lambda_context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/task/lambda_function.py", line 21, in lambda_handler
invoke(test_cryptography)
File "/var/task/lambda_function.py", line 35, in invoke
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/var/task/lambda_function.py", line 49, in test_cryptography
from cryptography.fernet import Fernet
File "/opt/python/cryptography/fernet.py", line 14, in <module>
from cryptography.exceptions import InvalidSignature
File "/opt/python/cryptography/exceptions.py", line 9, in <module>
from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
pyo3_runtime.PanicException: Python API call failed
RequestId: fc62938a-dfd5-4b22-9131-d3413233d6d4 Error: Runtime exited with error: exit status 1
Runtime.ExitError

[ChrisHills463]

I came here because the Lambda documentation still points to manylinux2014_x86_64 regardless of the runtime used. I hope this can get updated soon!