pncnmnp
commented
4 years ago Changing this line (in serving.py) to - @bentoml.env(infer_pip_packages=True) did fix the dependency issue. However, the 503 error still persists.
Looking again into Heroku's log files, I found a new issue:
2020-09-27T07:30:27.520399+00:00 app[web.1]: File "/bento/PyTorchModel/serving.py", line 14, in
2020-09-27T07:30:27.520399+00:00 app[web.1]: from config import config 2020-09-27T07:30:27.520400+00:00 app[web.1]: File "/bento/PyTorchModel/config.py", line 3, in 2020-09-27T07:30:27.520400+00:00 app[web.1]: class config: 2020-09-27T07:30:27.520401+00:00 app[web.1]: File "/bento/PyTorchModel/config.py", line 13, in config 2020-09-27T07:30:27.520401+00:00 app[web.1]: do_lower_case=True 2020-09-27T07:30:27.520401+00:00 app[web.1]: File "/opt/conda/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 1425, in from_pretrained 2020-09-27T07:30:27.520402+00:00 app[web.1]: return cls._from_pretrained(*inputs, **kwargs) 2020-09-27T07:30:27.520402+00:00 app[web.1]: File "/opt/conda/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 1531, in _from_pretrained 2020-09-27T07:30:27.520403+00:00 app[web.1]: list(cls.vocab_files_names.values()), 2020-09-27T07:30:27.520404+00:00 app[web.1]: OSError: Model name '../model/' was not found in tokenizers model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-dbmdz-uncased, TurkuNLP/bert-base-finnish-cased-v1, TurkuNLP/bert-base-finnish-uncased-v1, wietsedv/bert-base-dutch-cased). We assumed '../model/' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.txt'] but couldn't find such vocabulary files at this path or url.
The issue is the config.py file and Heroku is unable to find the ../model directory and its contents. Checking the local copy of my BentoML model which got deployed as a docker image on Heroku, I can see the following file structure:
[13:06][20200926223007_B35B21] tree . . ├── bentoml-init.sh ├── bentoml.yml ├── docker-entrypoint.sh ├── Dockerfile ├── environment.yml ├── MANIFEST.in ├── python_version ├── PyTorchModel │ ├── artifacts │ │ ├── init.py │ │ └── ner.pt │ ├── bentoml.yml │ ├── config.py │ ├── dataset.py │ ├── init.py │ ├── model.py │ ├── serving.py │ └── utils.py ├── README.md ├── requirements.txt └── setup.py 2 directories, 19 files [13:06][20200926223007_B35B21]
It has not loaded the model directory and its 3 contents - config.json, pytorch_model.bin, and vocab.txt, although it was there on my local repository (which was saved by BentoML). I am unsure if we have to change the file structure or provide a special provision on BentoML.
The deployment on Heroku was a bumpy ride. Initially I encountered the following error:
I noticed a similar error for
Torch==1.6.0+cpu. As I was not using CUDA (#2), The following thread on Stack Overflow - Install PyTorch from requirements.txt fixed the issue.At this stage BentoML's
requirements.txtfor my latest model was in the following form:The deployment went fine without any issues. However, after making a prediction request:
I was getting a 503 Error. From Heroku's Log:
2020-09-26T15:37:11.745901+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=POST path="/predict" host=bentoml-her0ku-mtywmteynzm1mgo.herokuapp.com request_id=a971ef08-e573-487a-9e23-2f0f5ba0e87a fwd="49.32.63.210" dyno= connect= service= status=503 bytes= protocol=httpsHeroku Docs says the following about
H10 error- A crashed web dyno or a boot timeout on the web dyno will present this error.Inspecting the log file, I found the issue:
Transformers module was not installed. I believe this is due to the following line-
@bentoml.env(pip_packages=['torch', 'numpy', 'torchvision', 'scikit-learn'])BentoML is being explicitly asked to install the following four packages and not Transformers.
@bentoml.env(infer_pip_packages=True)should fix the issue.