search

micmurawski / poetry-plugin-lambda-build

Poetry plugin for building packages for serverless functions deployments like AWS Lambda, Google App Engine, Azure App Service, and more...
MIT License
39 stars 5 forks source link

`copy_to_container` fails if the project is large #32

Open toxygene opened 2 weeks ago

toxygene commented 2 weeks ago

Assuming I understand the code correctly, it appears that the copy_to_container method creates a tar file containing the entire project, and copies it to the container. If the project contains any large files (in my case, I had a cdk.out directory that was ~2.5G), the container.put_archive call fails with a mysterious ('Connection aborted.', OSError(22, 'Invalid argument')) error. Best I've been able to determine, this is due to the filesize exceeding some limit imposed by the OS (MacOS Sonoma 14.6.1, in my case).

It would be helpful if this plugin supported something similar to .dockerignore or an exclude property in the pyproject.toml file. It looks like TarFile.add supports a filter argument, which looks like a good starting point.

Attached is the stacktrace that was output when the OSError occurred.

stacktrace.txt

micmurawski commented 2 weeks ago

Let me think about some solutions. I will comeback to you with a feature branch.

micmurawski commented 2 weeks ago

@toxygene I am still working on finishing touches, but you may check how the feature works for you by installing it from the feature branch

poetry self add git+ssh://git@github.com/micmurawski/poetry-plugin-lambda-build@feat/implement-ignore-files

if you run poetry lambda-build --help, you should notice a new field copy-to-container-ignore you can provide it as an argument of CLI execution poetry lambda-build --copy-to-container-ignore, or in pypoject.toml

copy-to-container-ignore = ["cdk.out"]

I am curious about your feedback. Let me know if that works for you.