search

alexanderluiscampino / lambda-layers

Several layers for AWS Lambda
32 stars 15 forks source link

Pyodbc layer #1

Open reidca opened 4 years ago

reidca commented 4 years ago

Spent a very long time trying to get PyOdbc working in Lambda (to connect to an MS Sql DB) without success due to dependencies on unix odbc not being present in Lambda runtime.

Thankfully came across your layer after several hours of searching and this has helped me get this working. Thank you for creating and sharing it.

Do you have any instructions on how you did it? I am conscious that as versions of Python or the MS driver change this may no longer work and I am interested in knowing how to recreate it.

Thanks

Carl

alexanderluiscampino commented 4 years ago

Hi @reidca ,

thats actually a great idea, and have been wanting to create some serverless deployment file to handle just that. Just haven't had the time. I will create a branch soon and embrace that project., willing to take on helpers and contributors.

The way I did it was very much trial and error. I create a conda environment inside this docker image

I installed the pyodbc library using pip3, Then I located the compiled libraries I needed (so files) and moved them from the linux root to where pyodbc was installed, so everything was locally found. on the odbcinst, there was a need to change the file pointer. Again, I did trial and error, see what broke, read the error message, located the needed file, moved it, re-deployed my layer, until it worked.

Ideally, want to create a bash script that does all of this, independent of the python version and/or the odbc version (currently 17)

Im sorry, cant be super detailed, since it has been a while. But really want to make this a better repo so people can use it.

reidca commented 4 years ago

Thanks for replying. I could not believe how difficult it was to get this working in Lambda. There are a lot of posts with contradictory instructions and lots of mixed versions. I am far from an expert in python packaging, is there a reason why the dependencies could not be put into a pip package ?

In any case, once again, I am very grateful for you sharing your hard work. Really saved me.

If you are able to create either a runbook or automation script to help update this when versions inevitably change that would be very useful.

Would you also consider publishing this as a public layer and maybe getting it listed on here (which is actually linked from AWS official documentation).

Thanks

C0rvus commented 4 years ago

Hi, I also want to say you, thank you so much for providing this zip file in your repo! It was so smooth, importing it in lambda layer and it worked right out of the boxed.

dariosuardiaza1 commented 1 year ago

@alexanderluiscampino dude you absolutely positively f'ing killed it with this!! you saved me countless hours of trial/error not to mention the satisfaction of bypassing this problem in one shot, thanks a lot for sharing your hard work with us!! totally appreciate it!!

cheers mate! Dario

alexanderluiscampino commented 1 year ago

@dariosuardiaza1 pleasure to serve! keep up the good vibes my man!

aburela commented 1 year ago

I am trying to create the pyodbc layer for python3.10 . I am getting the "libodbc.so.2: cannot open shared object file: No such file or directory" even the files exists in lib folder. image image image

can someone help me with the issue ?

alexanderluiscampino commented 8 months ago

@aburela Each python version needs to be built specifically. Do you have a link for your build?