search

NicolasHug / Surprise

A Python scikit for building and analyzing recommender systems
http://surpriselib.com
BSD 3-Clause "New" or "Revised" License
6.34k stars 1.01k forks source link

pip installation with Scikit-surprise in AWS sam Error #393

Open thabied opened 3 years ago

thabied commented 3 years ago

Description

I am trying to deploy a customer recommendation API on AWS Lambda using the sam CLI and docker following this tutorial: https://sejalv.medium.com/containerized-ml-deployment-with-aws-lambda-680540fb92f4

I am using a base python image from aws's public ecr and I've added scikit-surprise and numpy in my requirements.txt file as stated in your installation doc but when I run $ sam build I get the following error message:

Using legacy setup.py install for scikit-surprise, since package 'wheel' is not installed.
Installing collected packages: pytz, six, python-dateutil, numpy, pandas, joblib, scipy, scikit-surprise, urllib3, chardet, certifi, idna, requests, threadpoolctl, scikit-learn
    Running setup.py install for scikit-surprise: started
    Running setup.py install for scikit-surprise: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /var/lang/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-6gksgbo5/scikit-surprise/setup.py'"'"'; __file__='"'"'/tmp/pip-install-6gksgbo5/scikit-surprise/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-xrkjvq3d/install-record.txt --single-version-externally-managed --compile --install-headers /var/lang/include/python3.7m/scikit-surprise
         cwd: /tmp/pip-install-6gksgbo5/scikit-surprise/
    Complete output (53 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/surprise
    copying surprise/dump.py -> build/lib.linux-x86_64-3.7/surprise
    copying surprise/builtin_datasets.py -> build/lib.linux-x86_64-3.7/surprise
    copying surprise/accuracy.py -> build/lib.linux-x86_64-3.7/surprise
    copying surprise/dataset.py -> build/lib.linux-x86_64-3.7/surprise
    copying surprise/reader.py -> build/lib.linux-x86_64-3.7/surprise
    copying surprise/__main__.py -> build/lib.linux-x86_64-3.7/surprise
    copying surprise/trainset.py -> build/lib.linux-x86_64-3.7/surprise
    copying surprise/__init__.py -> build/lib.linux-x86_64-3.7/surprise
    copying surprise/utils.py -> build/lib.linux-x86_64-3.7/surprise
    creating build/lib.linux-x86_64-3.7/surprise/model_selection
    copying surprise/model_selection/split.py -> build/lib.linux-x86_64-3.7/surprise/model_selection
    copying surprise/model_selection/validation.py -> build/lib.linux-x86_64-3.7/surprise/model_selection
    copying surprise/model_selection/__init__.py -> build/lib.linux-x86_64-3.7/surprise/model_selection
    copying surprise/model_selection/search.py -> build/lib.linux-x86_64-3.7/surprise/model_selection
    creating build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/algo_base.py -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/knns.py -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/predictions.py -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/baseline_only.py -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/__init__.py -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/random_pred.py -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    running egg_info
    writing scikit_surprise.egg-info/PKG-INFO
    writing dependency_links to scikit_surprise.egg-info/dependency_links.txt
    writing entry points to scikit_surprise.egg-info/entry_points.txt
    writing requirements to scikit_surprise.egg-info/requires.txt
    writing top-level names to scikit_surprise.egg-info/top_level.txt
    reading manifest file 'scikit_surprise.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'scikit_surprise.egg-info/SOURCES.txt'
    copying surprise/similarities.c -> build/lib.linux-x86_64-3.7/surprise
    copying surprise/similarities.pyx -> build/lib.linux-x86_64-3.7/surprise
    copying surprise/prediction_algorithms/co_clustering.c -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/co_clustering.pyx -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/matrix_factorization.c -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/matrix_factorization.pyx -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/optimize_baselines.c -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/optimize_baselines.pyx -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/slope_one.c -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    copying surprise/prediction_algorithms/slope_one.pyx -> build/lib.linux-x86_64-3.7/surprise/prediction_algorithms
    running build_ext
    building 'surprise.similarities' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/surprise
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/var/lang/lib/python3.7/site-packages/numpy/core/include -I/var/lang/include/python3.7m -c surprise/similarities.c -o build/temp.linux-x86_64-3.7/surprise/similarities.o
    unable to execute 'gcc': No such file or directory
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /var/lang/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-6gksgbo5/scikit-surprise/setup.py'"'"'; __file__='"'"'/tmp/pip-install-6gksgbo5/scikit-surprise/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-xrkjvq3d/install-record.txt --single-version-externally-managed --compile --install-headers /var/lang/include/python3.7m/scikit-surprise Check the logs for full command output.
WARNING: You are using pip version 20.1.1; however, version 21.1.2 is available.
You should consider upgrading via the '/var/lang/bin/python3.7 -m pip install --upgrade pip' command.

Build Failed
Error: HelloWorldFunction failed to build: The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1

Steps/Code to Reproduce

You need an aws account and aws-cli installed

Find a directory you want to work from and make sure docker is running and you're logged in:

pip install aws-sam-cli

sam init

select options 1 (template) then 2 (base image) then 5 (python3.7) - this will create a project template folder in this directory

add scikit-surprise and numpy to requirements.txt file and then run

sam build

Expected Results

I tried adding wheel to the requirements.txt file but it's still not working.

Please let me know if there's any other information you need from me

Actual Results

Versions

Darwin-20.3.0-x86_64-i386-64bit public.ecr.aws/lambda/python:3.7 as base - python 3.7 ecr base image surprise 1.1.1

NicolasHug commented 2 years ago

Hi @thabied , sorry I'm really not familiar with AWS but to install surprise you can either install a binary from conda, OR you'll have to build from source (e.g. when installing with pip install). That will require a compiler, Cython, etc (see our building from source instructions)

NicolasHug commented 2 years ago

Hi @thabied , sorry I'm really not familiar with AWS but to install surprise you can either install a binary from conda, OR you'll have to build from source (e.g. when installing with pip install). That will require a compiler, Cython, etc (see our building from source instructions)

sbrnaderi commented 1 year ago

Hi @thabied, I had the same issue and installing gcc on my Docker image that I use with AWS sam worked fine for me. I added this line to my Dockerfile before installing Surprise: RUN yum install gcc -y

I am deploying my Lambda function as a container image. See here for an example.