search

szymonmaszke / torchlambda

Lightweight tool to deploy PyTorch models to AWS Lambda
MIT License
125 stars 3 forks source link

torchlambda lacks non-source files in distributed package #2

Closed streicherlouw closed 4 years ago

streicherlouw commented 4 years ago

Hi.

I have been trying hard to follow the torchlambda example, but have simply come up against a hard stop that I cannot bridge. If you can offer any help, it would be much appreciated.

My installation is as such: I am working in a conda environment called fastai2, with torch 1.5.0 (full pip list below) and docker version 19.03.8.

I installed torchlambda with "pip3 install torchlambda". (I have also tried with "pip install --user torchlambda", which fails when executing in exactly the same later later)

(fastai2) streicher@MLPC:~$ pip3 install torchlambda
Collecting torchlambda
  Using cached torchlambda-1590904228-py3-none-any.whl (30 kB)
Requirement already satisfied: Cerberus>=1.3.2 in ./.local/lib/python3.7/site-packages (from torchlambda) (1.3.2)
Requirement already satisfied: PyYAML>=5.3 in /home/linuxbrew/.linuxbrew/lib/python3.7/site-packages (from torchlambda) (5.3.1)
Requirement already satisfied: setuptools in /home/linuxbrew/.linuxbrew/lib/python3.7/site-packages (from Cerberus>=1.3.2->torchlambda) (46.0.0)
Installing collected packages: torchlambda
Successfully installed torchlambda-1590904228
(fastai2) streicher@MLPC:~$

I then created the model as described on github:

(fastai2) streicher@MLPC:~$ python
Python 3.7.7 (default, Mar 26 2020, 15:48:22) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import torchvision
>>> model = torchvision.models.resnet18()
>>> torch.jit.script(model).save("model.ptc")
>>> quit()
(fastai2) streicher@MLPC:~$ ls *.ptc
model.ptc

Here starts the problems..

(fastai2) streicher@MLPC:~$ torchlambda settings
torchlambda:: Started creating YAML settings at /home/streicher/torchlambda.yaml.
torchlambda:: Started copying YAML source code
cp: cannot stat './templates/settings/torchlambda.yaml': No such file or directory
torchlambda:: Error: Failed during copying YAML source code
(fastai2) streicher@MLPC:~$ stat ./templates/settings/torchlambda.yaml
stat: cannot stat './templates/settings/torchlambda.yaml': No such file or directory

So I create the file manually using the example

(fastai2) streicher@MLPC:~$ vi torchlambda.yaml
(fastai2) streicher@MLPC:~$ cat torchlambda.yaml 
---
input:
  shape: [1, 3, width, height]
  type: byte
  cast: float
  divide: 255
normalize:
  means: [0.485, 0.456, 0.406]
  stddevs: [0.229, 0.224, 0.225]
return:
  result:
    operations: argmax
    type: int
    name: label
    item: true
(fastai2) streicher@MLPC:~$

Here comes the hard problem... I then try the template step, but it fails in a way that I cannot get around

(fastai2) streicher@MLPC:~$ torchlambda template --yaml torchlambda.yaml
torchlambda:: Started creating C++ scheme at ./torchlambda.
torchlambda:: Started reading YAML settings.
torchlambda:: Finished reading YAML settings.
torchlambda:: Started validating YAML settings.
torchlambda:: Error during YAML validation:
normalize:
- means:
  - 'means field''s shape is not broadcastable to provided input shape: [1, 3, ''width'',
    ''height'']'
  stddevs:
  - 'stddevs field''s shape is not broadcastable to provided input shape: [1, 3, ''width'',
    ''height'']'
(fastai2) streicher@MLPC:~$

Do you have any idea what I am doing wrong? I would really love to give tochlambda a try. I currently use the pytorch 1.1 ARN layer in lambda, and it takes fully 18 seconds to start up from cold...

Thanks.

Additional information:

My docker seems to be ok:

(fastai2) streicher@MLPC:~$ docker pull szymonmaszke/torchlambda:v1.5.0
v1.5.0: Pulling from szymonmaszke/torchlambda
a3f8e652bdc4: Pull complete 
8f876fde9c06: Pull complete 
cd0ee434189a: Pull complete 
Digest: sha256:c657e856e0b3f01cd7a8c3b3f603f94b0f45541ad8851afa056c988aaf59efa5
Status: Downloaded newer image for szymonmaszke/torchlambda:v1.5.0
docker.io/szymonmaszke/torchlambda:v1.5.0

(fastai2) streicher@MLPC:~/torchlambda$ docker image list
REPOSITORY                 TAG                                   IMAGE ID            CREATED             SIZE
szymonmaszke/torchlambda   v1.5.0                                d2496183b359        20 hours ago        711MB
samcli/lambda              python3.6-355e75366d87dabec462ddf97   ac2f11a96ba6        3 weeks ago         991MB
lambci/lambda              python3.6                             46852491e8e1        3 weeks ago         882MB
ubuntu                     latest                                1d622ef86b13        5 weeks ago         73.9MB
hello-world                latest                                bf756fb1ae65        5 months ago        13.3kB

(fastai2) streicher@MLPC:~$ docker run szymonmaszke/torchlambda
torchlambda:: Building AWS Lambda .zip package.
torchlambda:: Compilation flags: 
torchlambda:: Final build arguments: -DBUILD_SHARED_LIBS=OFF -DAWS_COMPONENTS=
-- The CXX compiler identification is GNU 7.3.1
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Torch: /usr/local/lib/libtorch.a  
-- Found CURL: /usr/lib64/libcurl.so (found version "7.61.1") 
-- Found AWS SDK for C++, Version: 1.7.344, Install Root:/usr/local, Platform Prefix:, Platform Dependent Libraries: pthread;crypto;ssl;z;curl
-- Configuring done
-- Build files have been written to: /usr/local/build
CMake Error at CMakeLists.txt:34 (add_executable):
  No SOURCES given to target: torchlambda
torchlambda:: App size:
du: cannot access '/usr/local/build/torchlambda': No such file or directory
torchlambda:: Zipped app size:
du: cannot access '/usr/local/build/torchlambda.zip': No such file or directory
torchlambda:: Deployment finished successfully.(fastai2) streicher@MLPC:~$

And the pip installation sees to be ok:

(fastai2) streicher@MLPC:~/aws-lambda$ pip3 list
Package                                    Version     
------------------------------------------ ------------
adal                                       1.2.3       
arrow                                      0.15.6      
attrs                                      19.3.0      
aws-lambda-builders                        0.8.0       
aws-sam-cli                                0.48.0      
aws-sam-translator                         1.22.0      
azure-cognitiveservices-search-imagesearch 2.0.0       
azure-common                               1.1.25      
beautifulsoup4                             4.9.0       
binaryornot                                0.4.4       
blis                                       0.4.1       
boto3                                      1.13.4      
botocore                                   1.16.4      
Bottleneck                                 1.3.2       
bs4                                        0.0.1       
catalogue                                  1.0.0       
Cerberus                                   1.3.2       
certifi                                    2020.4.5.1  
cffi                                       1.14.0      
chardet                                    3.0.4       
chevron                                    0.13.1      
click                                      7.1.2       
cookiecutter                               1.6.0       
cryptography                               2.9.2       
cycler                                     0.10.0      
cymem                                      2.0.3       
dateparser                                 0.7.4       
docker                                     4.2.0       
docutils                                   0.15.2      
fastcore                                   0.1.17      
fastprogress                               0.2.3       
Flask                                      1.0.4       
future                                     0.18.2      
graphviz                                   0.14        
idna                                       2.9         
importlib-metadata                         1.6.0       
isodate                                    0.6.0       
itsdangerous                               1.1.0       
Jinja2                                     2.11.2      
jinja2-time                                0.2.0       
jmespath                                   0.9.5       
joblib                                     0.14.1      
jsonschema                                 3.2.0       
kiwisolver                                 1.2.0       
MarkupSafe                                 1.1.1       
matplotlib                                 3.2.1       
msrest                                     0.6.13      
msrestazure                                0.6.3       
murmurhash                                 1.0.2       
numexpr                                    2.7.1       
numpy                                      1.18.4      
nvidia-ml-py3                              7.352.0     
oauthlib                                   3.1.0       
onnx                                       1.7.0       
packaging                                  20.3        
pandas                                     1.0.3       
Pillow                                     7.1.2       
pip                                        20.0.2      
plac                                       1.1.3       
poyo                                       0.5.0       
preshed                                    3.0.2       
protobuf                                   3.12.2      
pycparser                                  2.20        
PyJWT                                      1.7.1       
pyparsing                                  2.4.7       
pyrsistent                                 0.16.0      
python-dateutil                            2.8.0       
pytz                                       2020.1      
PyYAML                                     5.3.1       
regex                                      2020.4.4    
requests                                   2.23.0      
requests-oauthlib                          1.3.0       
s3transfer                                 0.3.3       
scikit-learn                               0.22.2.post1
scipy                                      1.4.1       
serverlessrepo                             0.1.9       
setuptools                                 46.0.0      
six                                        1.14.0      
soupsieve                                  2.0         
spacy                                      2.2.4       
srsly                                      1.0.2       
thinc                                      7.4.0       
tomlkit                                    0.5.8       
torch                                      1.5.0       
torchlambda                                1590904228  
torchvision                                0.6.0       
tqdm                                       4.46.0      
typing-extensions                          3.7.4.2     
tzlocal                                    2.0.0       
urllib3                                    1.25.9      
wasabi                                     0.6.0       
websocket-client                           0.57.0      
Werkzeug                                   1.0.1       
wheel                                      0.34.2      
whichcraft                                 0.6.1       
zipp                                       3.1.0
szymonmaszke commented 4 years ago

Hi, thanks for the report, I've confirmed the bugs you are talking about. Will hit you up as soon as the problem is fixed.

szymonmaszke commented 4 years ago

@streicherlouw fixed, reinstall torchlambda, both settings and template subcommands should work fine now, please report whether it's fine now on your side as well.

streicherlouw commented 4 years ago

Thank you for your quick response, I can confirm that the scripts work perfectly now :-)