search

serverless / serverless-python-requirements

⚡️🐍📦 Serverless plugin to bundle Python packages
MIT License
1.11k stars 291 forks source link

Numpy/Pandas not properly bundled on Windows #329

Open gdugernier opened 5 years ago

gdugernier commented 5 years ago

Hello,

First I would like to apologize for the lack of technical info to help solve this issue, but I'm not very proficient in anything python related. I'll do the best I can, though I think anything within my work is unrelated to the issue.

What went wrong?

I'm deploying a Lambda function in python which uses Pandas (and thus Numpy) and Sqlalchemy with Serverless through CodeBuild. The CodeBuild server is Linux and everything is working perfectly.

I now take the same project and deploy it directly from my Windows machine. The deployment is done without error but when I try to use the Lambda function I get the following error:

Unable to import module 'function_name': Missing required dependencies ['numpy']

It is important to note the two projects are identical aside from name changes. The only difference is the OS on which the deployment is done.

I also noted a different in the function size at the end. The function deployed on Linux is 22.2 MB and the one deployed on Windows is 21 MB. There doesn't seem to be missing files inside either zip, but Pandas and Numpy are the ones of different sizes between the two versions (though the content structure is the same). I'm guessing it's due to the Windows/Linux version of the dependencies being not entirely the same?

Additional information:

Severless: 1.35 Serverless-python-requirements: Latest Python: 2.7

dschep commented 5 years ago

Are you using the dockerizePip option? That is a requirement for deploying packages like numpy from mac os & windows.

euler2718 commented 5 years ago

Yeah what I did was include the linux (because thats the OS my lambda sits on) precompiled binaries in the in their respective folders at the same level as my lambda: https://medium.com/@korniichuk/lambda-with-pandas-fd81aa2ff25e

works too, but the dockerizePip is the idea, or layers with that.