No module named 'snappy._snappy_cffi' on AWS lambda

[AviEt]

I create a function using python-snappy to deploy on AWS Lambda. I followed this tutorial and basically did the following (TL;DR of the tutorial commands):

pip install --target ./package python-snappy
cd package
zip -r ../my-deployment-package.zip .
cd ..
zip -g my-deployment-package.zip lambda_function.py

When running my function it fails on import snappy line with:

Response
{
  "errorMessage": "Unable to import module 'lambda_function': No module named 'snappy._snappy_cffi'",
  "errorType": "Runtime.ImportModuleError",
  "stackTrace": []
}

My function file tree looks like the following:

my-function
--| python_snappy_0.6.1.dist-info
---- AUTHORS
---- INSTALLERS
---- <Rest of files>
--| snappy
---- __init__.py
---- __main__.py
---- <Rest of files
-- lambda_function.py

I looked at issue #52 (That was mentioned in another similiar issue to have solved the problem) but I'm not sure how to create the libsnappy.so.1 file (using mac M1 if it's relevant) or if there's another way to solve this issue.

[AviEt]

So, doing the exact same flow on a linux machine solved the issue. Guess the problem is the underlying binaries compilation

[chris1248]

I have the same problem. Why is this issue being ignored?

[AviEt]

@chris1248 - Have you seen my 2nd comment? I did the exact same steps on a linux machine and it worked. Guess it's an M1 chip issue

[chris1248]

Tried using the approach listed in the readme, and it doesn't work on a Mac M2 chip:

python test_snappy.py
Traceback (most recent call last):
  File "/..../venv/lib/python3.10/site-packages/snappy/snappy.py", line 48, in <module>
    from ._snappy import UncompressError, compress, decompress, \
ImportError: dlopen(/...i/venv/lib/python3.10/site-packages/snappy/_snappy.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '_snappy_compress'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/...i/src/test_snappy.py", line 1, in <module>
    import snappy
  File "/.../venv/lib/python3.10/site-packages/snappy/__init__.py", line 3, in <module>
    from .snappy import (
  File "/.../venv/lib/python3.10/site-packages/snappy/snappy.py", line 51, in <module>
    from .snappy_cffi import UncompressError, compress, decompress, \
  File "/.../venv/lib/python3.10/site-packages/snappy/snappy_cffi.py", line 3, in <module>
    from ._snappy_cffi import ffi, lib
ModuleNotFoundError: No module named 'snappy._snappy_cffi'

Brew info for snappy:

brew info snappy
==> snappy: stable 1.1.9 (bottled), HEAD
Compression/decompression library aiming for high speed
https://google.github.io/snappy/
/opt/homebrew/Cellar/snappy/1.1.9 (18 files, 164.7KB) *
  Poured from bottle on 2022-11-01 at 05:36:22
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/snappy.rb
License: BSD-3-Clause
==> Dependencies
Build: cmake ✘, pkg-config ✔
==> Options
--HEAD
        Install HEAD version
==> Analytics
install: 96,137 (30 days), 283,620 (90 days), 1,159,818 (365 days)
install-on-request: 1,526 (30 days), 4,596 (90 days), 23,338 (365 days)
build-error: 2 (30 days)

[martindurant]

I recommend using conda to install snappy, exactly because of these build problems. Failing that, the cramjam library provides wheels for all python platforms, and includes a version of snappy which is often faster than this one.