Python: Container including all python files at rootDir

[cosmic-ascendant]

Given a project structure like so:

.
├── function_a
│   ├── lambda.py
│   ├── pyproject.toml
│   ├── sst.pyi
│   └── uv.lock
├── function_b
│   ├── lambda.py
│   ├── pyproject.toml
│   ├── sst.pyi
│   └── uv.lock
├── README.md
├── sst.config.ts
├── sst-env.d.ts
├── sst_tree.txt
└── tsconfig.json

with an sst config like

export default $config({
  app(input) {
    return {
      name: "aws-python-container-two-functions",
      removal: input?.stage === "production" ? "retain" : "remove",
      home: "local",
      providers: {
        aws: true
      }
    };
  },
  async run() {
    const functionA = new sst.aws.Function("FunctionA", {
      python: {
        container: true,
      },
      handler: "function_a/lambda.handler",
      runtime: "python3.11",
      url: true
    });

    const functionB = new sst.aws.Function("FunctionB", {
      python: {
        container: true,
      },
      handler: "function_b/lambda.handler",
      runtime: "python3.11",
      url: true
    });

    return {
      functionAUrl: functionA.url,
      functionBUrl: functionB.url
    }
  },
});

upon issuing sst deploy we find that the build output for FunctionA-src looks like:

FunctionA-src
├── Dockerfile
├── function_a
│   ├── lambda.py
│   ├── pyproject.toml
│   ├── resources.json
│   ├── sst.pyi
│   └── uv.lock
└── function_b
    ├── lambda.py
    └── sst.pyi

The result being that all lambda functions are copied into any other given lambda function, which expands the image size to be the union of all lambda functions. I believe the pattern to follow when seeking to include other functions or libraries would be to use copyFiles as discussed in https://github.com/sst/ion/issues/1139.

I believe the source of this behavior seems to originate in /platform/src/runtime/python.ts

export async function buildPythonContainer(
                // ...
        const { pythonFiles, rootDir } = await getPythonFiles(parsed.dir);
                // ...
)

Namely we are passing the directory into getPythonFiles instead of the handler filepath. However, I find that when I pass file instead of parsed.dir, some needed dependencies are then missing. So perhaps something more is needed here..

[cosmic-ascendant]

@walln Did you have a different approach lined up for this? Or should I try to PR a fix based on the current implementation. Thanks

[walln]

I've been toying with a few ways to get this working but have yet to come up with something I like; a few other issues have been created about working in monorepos that overlap with this issue. The main problem is resolving all the needed Python files and preserving their relative structure, which is non-trivial given all the ways I have seen people trying to make this work and how sst dev works. If you need an immediate fix, I suggest filing a PR, as I am unsure when I will have something ready.

[cosmic-ascendant]

I see, thanks. Understandable that a nice fix covering the myriad of ways people want to organize their project structure is proving difficult. I'm squeaking by for the moment needing only one python lambda container, but if I meet a wall I'll propose a PR. Perhaps could be helpful if we consolidate a thread on the various project structures and proposed benefits/shortcomings of approaches