RFC: Enable Python, Go, and others with `defineFunction`

[josefaidt]

Hey folks :wave: we're looking to add support for "custom" runtimes with defineFunction! This is a follow-up to the following issues, and serves as a request for feedback on how we're thinking of addressing the feature requests

• https://github.com/aws-amplify/amplify-backend/issues/1486
• https://github.com/aws-amplify/amplify-backend/issues/949

We are looking to bridge the gap between defineFunction and defining functions using CDK, which then require sideloading to Amplify resources with CDK (e.g. adding a Python function as a trigger for Storage's underlying S3 bucket). To achieve this, we're thinking about an escape hatch to wrap initialized CDK constructs with defineFunction:

import { defineFunctionWithAnyLanguage as defineFunction } from "@aws-amplify/backend"
import { Runtime } from "aws-cdk-lib/aws-lambda"
import { GoFunction } from "@aws-cdk/aws-lambda-go-alpha"
import { PythonFunction } from "@aws-cdk/aws-lambda-python-alpha"

// a regular function
defineFunction({
  name: "my-regular-function",
})

// a go function
defineFunction((scope) => {
  return new GoFunction(scope, "GoFunction", {
    entry: "app/cmd/api",
  })
})

// a python function
defineFunction((scope) => {
  return new PythonFunction(scope, "PythonFunction", {
    entry: '/path/to/my/function',
    runtime: Runtime.PYTHON_3_8,
  })
})

Many of the features of defineFunction today are specific to the TypeScript developer experience such as secrets, automatic secret resolution, typed environment variables, and sensible default configuration for bundling. Using this escape hatch would opt out of consuming these features.

However, this gives you a way to use Python, Go, or other runtimes with your Functions, and attach to other Amplify resources without CDK. This enables use cases such as:

• Python resolvers for Data models, custom queries
• Go resolvers for Data models, custom queries
• Python-based triggers for Storage

Let us know what you think!

[LukaASoban]

This looks great! Exactly what I was hoping for (similar way of defining the function itself)

[MarlonJD]

Hello @josefaidt. Awesome, it would be great!

Using this defineFunction in defineBackend like typescript would be perfect solution.

backend.ts:

export const sayHelloGoHandler = defineFunction((scope) => {
  return new GoFunction(scope, "GoFunction", {
    entry: "app/cmd/api",
  })
});

defineBackend({
  auth,
  data,
  sayHelloHandler,
});

It should be usable in data/resource.ts:

import { a, defineData, type ClientSchema } from "@aws-amplify/backend";
import { sayHelloGoHandler } from "../functions/say-hello-go/resource";

const schema = a.schema({
  // 1. Define your return type as a custom type
  EchoResponse: a.customType({
    content: a.string(),
    executionDuration: a.float(),
  }),

  sayHelloGo: a
    .query()
    // arguments that this query accepts
    .arguments({
      content: a.string(),
    })
    // return type of the query
    .returns(a.ref("EchoResponse"))
    // only allow signed-in users to call this API
    .authorization((allow) => [allow.authenticated()])
    // 3. set the function has the handler
    .handler(a.handler.function(sayHelloGoHandler)),
});

export type Schema = ClientSchema<typeof schema>;

export const data = defineData({
  schema,
  authorizationModes: {
    defaultAuthorizationMode: "userPool",
    apiKeyAuthorizationMode: {
      expiresInDays: 365,
    },
  },
});

[brandoncroberts]

This looks like a good proposal! I have a use case for this and would like to use it for a Python runtime

[MarlonJD]

I just created golang function for gen 2, but I did like this. It was easier than proposal, runtime default is still NODEJS 18 (NodeJS_18), working example is like this:

export const sayHelloGoHandler = defineFunction({
  name: "say-hello",
  entry: "./",
  runtime: "GO_1_X_PROVIDED_AL2023",
});

export const sayHelloPythonHandler = defineFunction({
  name: "say-hello-python",
  entry: "./",
  runtime: "PYTHON_3_8",
});

I edited runtime parameters options to:

     * Defaults to the oldest NodeJS LTS version. See https://nodejs.org/en/about/previous-releases
     * Other supported runtimes are:
     * - "NODEJS_16_X"
     * - "NODEJS_18_X"
     * - "NODEJS_20_X"
     * - "GO_1_X_PROVIDED_AL2023"
     * - "PYTHON_3_8"
     * - "PYTHON_3_9"
     * - "PYTHON_3_10"
     * - "PYTHON_3_11"
     * - "PYTHON_3_12"
     */

It can be usable in data:

EchoResponse: a.customType({
    content: a.string(),
    executionDuration: a.float(),
  }),

  sayHelloGo: a
    .query()
    // arguments that this query accepts
    .arguments({
      content: a.string(),
    })
    // return type of the query
    .returns(a.ref("EchoResponse"))
    // only allow signed-in users to call this API
    .authorization((allow) => [allow.authenticated(), allow.publicApiKey()])
    // 3. set the function has the handler
    .handler(a.handler.function(sayHelloGoHandler)),

Golang function creating and custom handler with data query is successful, but there is important issue on python function runtimes, on cdk it's saying docker will necessary on python runtime bundling, for this solution we have install docker for using python lambda functions, check it out here

What are you thinking about this solution? I can raise a PR for this now. @josefaidt @LukaASoban @brandoncroberts

[Arnoldguti]

I have a use case with Python runtime!

[MarlonJD]

I have a use case with Python runtime!

Please leave 👍 to my PR about this #1602. You can found information about python functions.