Failed to load psycopg in an uploaded function

[butla]

The function I've built and uploaded with lambda-uploader crashed with this error: Unable to import module 'kinesis_worker': /var/task/psycopg2/_psycopg.so: undefined symbol: PyUnicodeUCS4_DecodeUTF8

My requirements.txt:

credstash==1.14.0
psycopg2==2.7.3.2

My lambda.json:

{
  "name": "kinesis-worker",
  "description": "Takes events off the Kinesis queue and puts them into the DB.",
  "region": "us-east-1",
  "runtime": "python3.6",
  "handler": "kinesis_worker.dump_records_to_db",
  "role": "arn:aws:iam::609235665469:role/test_worker_role",
  "ignore": [
    "asd"
  ],
  "subscription": {
    "kinesis": {
      "stream": "arn:aws:kinesis:us-east-1:609235665469:stream/test_stream",
      "batch_size": 500
    }
  }
}

The function works with AWS when I'm building with my own scripts (created them because of #151):

# build.sh
SCRIPT_LOCATION=$(realpath $(dirname $0))
echo Building a lambda package in $BUILD_DIR

VENV_DIR=${SCRIPT_LOCATION}/lambda_venv
if [ -d $VENV_DIR ]; then
    echo Removing old ${VENV_DIR}
    rm -rf $VENV_DIR
fi
python3.6 -m venv $VENV_DIR
${VENV_DIR}/bin/pip install -r ${SCRIPT_LOCATION}/requirements.txt

cp -r ${VENV_DIR}/lib/python3.6/site-packages ${SCRIPT_LOCATION}
BUILD_DIR=${SCRIPT_LOCATION}/lambda_build
if [ -d $BUILD_DIR ]; then
    echo Removing old ${BUILD_DIR}
    rm -rf $BUILD_DIR
fi
mv ${SCRIPT_LOCATION}/site-packages ${BUILD_DIR}

cp ${SCRIPT_LOCATION}/*.py ${BUILD_DIR}
find ${BUILD_DIR} -name __pycache__ | xargs rm -rf

LAMBDA_PACKAGE=lambda_package.zip
(
    cd $BUILD_DIR
    zip -r ${BUILD_DIR}/${LAMBDA_PACKAGE} .
    mv $LAMBDA_PACKAGE $SCRIPT_LOCATION
)

echo Built $LAMBDA_PACKAGE in $SCRIPT_LOCATION

# upload.sh
SCRIPT_LOCATION=$(realpath $(dirname $0))
LAMBDA_PACKAGE=${SCRIPT_LOCATION}/lambda_package.zip

aws lambda update-function-code \
    --function-name arn:aws:lambda:us-east-1:609235665469:function:kinesis_worker \
    --zip-file fileb://${LAMBDA_PACKAGE}

My OS is Kubuntu 17.10

[Julio-Assis]

I'm having the same issue here, did you find a fix for it?

[jarosser06]

Looks like an issue with compiled shared libraries trying to run on a different target system. This doesn't appear to be an issue with the lambda-uploader as much as an issue with not building the virtualenv in an Amazon Linux based system.

There are a few workarounds, the easiest is to build the project in a local Amazon Linux container. If I have some time I'll try and write up a little example of how to go about this.

[butla]

@Julio-Assis I've dropped my ideas to use lambda-uploader because of this problem and because it didn't seem to fit our workflow.

We are using our own script that builds the Lambda package as a layer in a docker image extending lambci/lambda:build-python3.6 We then copy that package, from a started container, upload it to S3, and then set it up with Terraform.

We may publish details in a blog post at one point :-)

[butla]

@jarosser06 Oh, I didn't know that you need to use it from Amazon Linux. Guess it'd be good to put it in the readme.

[Julio-Assis]

One possible workaround that I have in mind is to change my database to mysql and then I would use pymysql rather than psycopg2. Do you think this will work?

[butla]

@Julio-Assis I think that building on Amazon Linux, or in lambci/lambda:build-python3.6 container would be a better idea. I don't think you'll be able to escape using any C extensions for long. Also, I'm a Postgres fan :)