Can't use Poetry in mixed runtime

[yusukegoto]

Are you certain it's a bug?

• [X] Yes, it looks like a bug

Are you using the latest plugin release?

• [X] Yes, I'm using the latest plugin release

Is there an existing issue for this?

• [X] I have searched existing issues, it hasn't been reported yet

Issue description

I got an error from sls package command when I mixed runtimes and enabled dockerizePip. This happens with poetry.

Service configuration (serverless.yml) content

pythonRequirements:
        dockerizePip: true
    mypython:
        runtime: python3.11
        hander: python/index.handler

    mynodejs:
        runtime: nodejs18.x
        handler: nodejs/index.handler

### Command name and used flags

sls package --verbose

### Command output

```shell
Packaging
before:package:createDeploymentArtifacts
Compiling to node18 bundle with esbuild...
Compiling with concurrency: Infinity
Compiling completed.
Zip service knox-api - 668.64 KB [75 ms]
Generating requirements.txt from poetry.lock
Parsed requirements.txt from pyproject.toml in /Users/yusukegoto/src/immo/knox/knox-api/.serverless/requirements.txt
Installing requirements from "/Users/yusukegoto/Library/Caches/serverless-python-requirements/d2318182abd0ad6c128797309cc391060ea3a28e2eb04ac79b8bb3879ef5634e_x86_64_slspyc/requirements.txt"
Docker Image: public.ecr.aws/sam/build-nodejs18.x:latest-x86_64
Using download cache directory /Users/yusukegoto/Library/Caches/serverless-python-requirements/downloadCacheslspyc
Running docker run --rm -v /Users/yusukegoto/Library/Caches/serverless-python-requirements/d2318182abd0ad6c128797309cc391060ea3a28e2eb04ac79b8bb3879ef5634e_x86_64_slspyc\:/var/task\:z -v /Users/yusukegoto/Library/Caches/serverless-python-requirements/downloadCacheslspyc\:/var/useDownloadCache\:z -u 0 public.ecr.aws/sam/build-nodejs18.x\:latest-x86_64 python -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache...
Environment: darwin, node 18.17.1, framework 3.38.0 (local), plugin 7.2.0, SDK 4.5.1
Credentials: Local, environment variables
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
Running "docker run --rm -v /Users/yusukegoto/Library/Caches/serverless-python-requirements/d2318182abd0ad6c128797309cc391060ea3a28e2eb04ac79b8bb3879ef5634e_x86_64_slspyc:/var/task:z -v /Users/yusukegoto/Library/Caches/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z -u 0 public.ecr.aws/sam/build-nodejs18.x:latest-x86_64 python -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache" failed with: "WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
/usr/bin/python: No module named pip"

### Environment information

```shell
Framework Core: 3.38.0 (local)
Plugin: 7.2.0
SDK: 4.5.1

I'm using27b70f4d6a7e43fd0e9711bbb56752fee2762901 to get a fixed patch for mixed runtimes.

[pgrzesik]

Hey @yusukegoto 👋 Thanks for reporting. Could you provide a minimal reproducible example that I could try testing out on my side? It would be really helpful 🙇

[yusukegoto]

@pgrzesik This is a link to my sample app. https://github.com/yusukegoto/sls-python-poetry-example

And I found out this happens at least with nodejs18.x. If I update two nodejs18.x in serverless.yml with nodejs20.x , package command succeeds 🤷‍♂️

[pgrzesik]

Thanks a lot @yusukegoto 🙇 I'll try to reproduce with your example and will report back 👍

[pgrzesik]

Thanks again @yusukegoto. I managed to reproduce the issue and understand what is causing it. The issue stems from the fact that when you don't use package.individually: true, then the docker image is based on the provider runtime, which in this case is nodejs18.x. The fact that it works with nodejs20.x is actually surprising to me, I guess maybe that image includes Python with pip for some reason?

I'm honestly not sure about the resolution here. The options I see:

1. Detect such situation and error out, suggesting to upgrade to python runtime on provider level. It's not ideal as having other runtime as default sounds valid.
2. Detect such situation and error out, suggesting to use package.individually: true in such cases. The downside here is that we potentially have multiple functions that use the same Python runtime that we could package toghether, not individually. But also, we could have different Python runtimes specified, in which we case we should force individual packaging.

All of these solutions do not sound amazing to be fair, I'm leaning towards checking if all Python functions have the same runtime and if yes, then using that one and erroring out only in case of a mismatch. Of course that would only apply if provider-level runtime is non-python.

What do you think @yusukegoto ?