ahupp / python-magic

A python wrapper for libmagic
Other
2.59k stars 280 forks source link

Having Trouble Building Serverless execution #311

Open NTC4818 opened 7 months ago

NTC4818 commented 7 months ago

Tried lots of internal remapping and building with libmagic, but no success. Seems it is looking for a fundamental part of the container that is not there, but I am not sure. I built the library with Ubuntu 22.04 and here is the main error I see when executing in AWS Lambda /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /var/task/lib/libmagic.so.1). I have pulled the libmagic.so.1 as recommended on other forms from an ec2 to be part of the execution.

ahupp commented 7 months ago

I assume you are the person who mailed me about this? I suspect what is happening is that the python version you're using is linked against an older libc than libmagic is built with. Even if that libc exists on the machine, you can't have to different versions in one process. Where is your python binary coming from?

NTC4818 commented 7 months ago

It is the official python binary from AWS version 3.8 but if I can get anything + 3.7 I’m good with that if anyone has a file working

On Mon, Jan 29, 2024 at 12:59 PM Adam Hupp @.***> wrote:

I assume you are the person who mailed me about this? I suspect what is happening is that the python version you're using is linked against an older libc than libmagic is built with. Even if that libc exists on the machine, you can't have to different versions in one process. Where is your python binary coming from?

— Reply to this email directly, view it on GitHub https://github.com/ahupp/python-magic/issues/311#issuecomment-1915276004, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR7JAFXI75KDC3ZDPXNAJ7TYQ7PQRAVCNFSM6AAAAABCMI3HGOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJVGI3TMMBQGQ . You are receiving this because you authored the thread.Message ID: @.***>

ddelange commented 5 months ago

^ for now, can you try the following install command and report back if it works?

pip install python-magic --force-reinstall --find-links https://github.com/ddelange/python-magic/releases/expanded_assets/0.4.28
NTC4818 commented 5 months ago

Awesome, sorry I missed that note. I will spin up an ec2 with python 3.9 and run the following command, prepare it as a lambda layer and place where I currently have python 3.8 version and test run. Do you think there is anything else I would need to change for that to be functional? Thanks a million

NTC4818 commented 5 months ago

Not quite the ticket. I built the layer as requested with the pip install command with pip 3.9 and then updated the brotlipy layer as well with 3.9 and received the following error. [ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': No module named 'brotli._brotli' Traceback (most recent call last) Not sure if this has been seen in the past layer development and maybe the packaging of the layers need to be packaged together. Thanks

ddelange commented 5 months ago

brotlipy is not maintained, rather go for brotli: you can install both in a single layer:

pip install brotli python-magic --force-reinstall --find-links https://github.com/ddelange/python-magic/releases/expanded_assets/0.4.28
NTC4818 commented 5 months ago

Still Not reading as it was in python 3.7, any ideas why it might be acting differently. It is running in aws lambda environment