sst / ion

SST v3
https://sst.dev
MIT License
2.13k stars 250 forks source link

Python: .venv in function handler dir conflicting with Docker .venv #1216

Open cosmic-ascendant opened 1 week ago

cosmic-ascendant commented 1 week ago

Using the aws-python-container example, if our project looks like this:

.
├── README.md
├── src
│   ├── pyproject.toml
│   ├── python.py
│   ├── uv.lock
│   └── .venv
├── sst.config.ts
├── sst-env.d.ts
└── tsconfig.json

Then we find that .venv is copied into the build container and interferes with the uv logic that installs the dependencies from pyproject.toml.

I believe this can be fixed within /platform/src/runtime/python.ts

async function getPythonFiles(
    filePath: string,
): Promise<{ pythonFiles: string[]; rootDir: string }> {

                       // ...
            for (const entry of entries) {
                const entryPath = path.join(currentPath, entry.name);

                if (entry.isDirectory()) {
                    if (entry.name === "__pycache__" || entry.name === ".venv") {
                        // Skip directories named "__pycache__" or ".venv"
                        continue;
                    }
                           // ...

I tested this on the python-container example and seemed to resolve. @walln

JacksonBowe commented 1 week ago

@cosmic-ascendant I think that example is showcasing a bad file structure because .venv should really be in the root directory. I've spent the last few days heavily experimenting with python project structures. Full disclaimer this was in SSTv2, using uv, in preparation for migrating to SSTv3.

The workflow + structure that I ended up with was:

.venv <-- top level, this facilitates local dev
app/ <-- create with "uv init <name>" this will make a uv workspace
    main.py
    pyproject.toml <-- the dependencies needed by this deployment, which will get used when building
sst.config.st
tsconfig.json
sst-env.d.ts
pyproject.toml <-- just the default from "uv init"
uv.lock

In the root of your project you can then run uv add .\app\ and uv sync which will recreate your .venv and install the packages from app/pyproject.toml. The downside to this is that there is no app/uv.lock. I guess you could copy it in with copyFiles.

Anyway, that's my 2 cents.

cosmic-ascendant commented 5 days ago

In my case, I needed and wanted to keep my virtual env's separate for each lambda, for example:

├── function_a
│   ├── main.py
│   └── pyproject.toml
├── function_b
│   ├── main.py
│   └── pyproject.toml
├── package.json
├── package-lock.json
├── README.md
├── sst.config.ts
└── tsconfig.json

That said, perhaps there is a solution using uv workspaces, but I'm not sure yet.