Feature request: Add support for pyproject.toml

[pkarakal]

Describe your idea/feature/enhancement

As the Python ecosystem moves towards pyproject.toml as the standard for defining build system requirements and managing dependencies, it would be highly beneficial if there was a lambda builder to recognize and handle this file natively. Using setup.py is gradually being phased out in favor of pyproject.toml which modern package managers (like Poetry) support to align with PEP 517 and PEP 518 standards.

Currently, we either have to manually export our dependencies to a requirements.txt file, or have makefile as the build method in the template.yaml where the makefile is exporting the deps to a requirement.txt before preparing the function:

# template.yaml
Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.12
    Metadata:
      BuildMethod: makefile

build-HelloWorldFunction:
  poetry export -f requirements.txt --output requirements.txt --without-hashes
  pip install --target "$(ARTIFACTS_DIR)" -r requirements.txt
  cp -r *.py $(ARTIFACTS_DIR)

This can be tedious when having to deal with multiple functions.

Proposal

Implement a builder, similar to python_pip that parses pyproject.toml, recognizes the build-backend defined in pyproject.toml (such as poetry or flit) and handles dependencies installation accordingly. This way when invoking sam build the existence of pyproject.toml is also taken into consideration (and not only requirements.txt and setup.py). Based on this, it should be able to select the appropriate builder to use.

[hawflau]

Hey @pkarakal, thanks for raising the issue. I will bring up with team for this feature request

[mrburrito]

Going to + :infinity: this. Between this project and the SAM CLI, I think people have been asking for this capability for over five years now. (see this open ticket on aws-sam-cli) I know it was a huge frustration on a project I worked on a year ago because we abandoned moving to poetry specifically because aws-lambda-builders, and by extension SAM, didn't have native support for pyproject.toml and it wasn't worth the hassle of trying to implement workarounds.

The sooner you can get a pyproject.toml workflow in place, the better!

[Dilski]

PEP 735 has just been accepted, which defines a new standard for defining dependency groups in pyproject.toml. This lets you define groups, and reference other groups. I'm imagining (once the dust settles, and tooling like poetry, uv, pip[?] catch on) we could do something like this:

[dependency-groups]
main = ["aws-lambda-powertools"]
database = ["psycopg3"]
MyLambdaFunction1 = [{include-group = "main"}, {include-group = "database"}]
MyLambdaFunction2 = [{include-group = "main"}]
test = ["pytest", "coverage"]
dev = [{include-group = "main"}, {include-group = "database"}, {include-group = "test"}]

I'm not sure the limitations of aws-lambda-builders, but it would be neat to have a single pyproject.toml at the root of the project that is picked up which will use a dependency group matching the logical ID