search

calpoly-csai / api

Official API for the NIMBUS Voice Assistant accessible via HTTP REST protocol.
https://nimbus.api.calpolycsai.com/
GNU General Public License v3.0
9 stars 4 forks source link

dockerfile works with ubuntu #98

Closed mfekadu closed 4 years ago

mfekadu commented 4 years ago

docker build

(api) ➜  api git:(mf-pipenv) ✗ make docker
invoke docker
docker username? mfekadu
hey run this to make life easier...
export TASKS_DOCKER_USERNAME=mfekadu
make first sure to run...
source .export_env_vars

$ docker build --build-arg DATABASE_HOSTNAME --build-arg DATABASE_PASSWORD --build-arg DATABASE_USERNAME --build-arg DATABASE_NAME --build-arg PYDRIVE_CLIENT_ID --build-arg PYDRIVE_CLIENT_SECRET --build-arg GOOGLE_DRIVE_CREDENTIALS --build-arg GOOGLE_DRIVE_FOLDER_ID --build-arg GOOGLE_CLOUD_NLP_CREDENTIALS --build-arg GOOGLE_CLOUD_NLP_MODEL_NAME -t "mfekadu/nimbus" .

Sending build context to Docker daemon  807.9MB
Step 1/32 : FROM ubuntu:latest
 ---> 72300a873c2c
Step 2/32 : RUN apt update
 ---> Using cache
 ---> 1ee360506a5b
Step 3/32 : RUN apt-get update   && apt-get install -y python3-pip python3-dev   && cd /usr/local/bin   && ln -s /usr/bin/python3 python   && pip3 install --upgrade pip
 ---> Using cache
 ---> c735d307d0ad
Step 4/32 : ADD requirements.txt /nimbus/requirements.txt
 ---> Using cache
 ---> dd08805e7479
Step 5/32 : RUN pip install -r /nimbus/requirements.txt
 ---> Using cache
 ---> 834de863ddfd
Step 6/32 : ADD . /nimbus
 ---> f0907b545ae6
Step 7/32 : ARG DATABASE_HOSTNAME
 ---> Running in deccb4bc94ed
Removing intermediate container deccb4bc94ed
 ---> a193318c442f
Step 8/32 : ARG DATABASE_PASSWORD
 ---> Running in 563e6ee43307
Removing intermediate container 563e6ee43307
 ---> 4a7e63ca082a
Step 9/32 : ARG DATABASE_USERNAME
 ---> Running in 911c38807bdc
Removing intermediate container 911c38807bdc
 ---> d2bd04f5c9cc
Step 10/32 : ARG DATABASE_NAME
 ---> Running in d85996cdb153
Removing intermediate container d85996cdb153
 ---> 3567fecef34e
Step 11/32 : ARG PYDRIVE_CLIENT_ID
 ---> Running in 7a1c2952c55e
Removing intermediate container 7a1c2952c55e
 ---> 533f944d96f9
Step 12/32 : ARG PYDRIVE_CLIENT_SECRET
 ---> Running in 9872599b9797
Removing intermediate container 9872599b9797
 ---> 28ff08b252fa
Step 13/32 : ARG GOOGLE_DRIVE_CREDENTIALS
 ---> Running in 99d5def24b99
Removing intermediate container 99d5def24b99
 ---> e5523833b10a
Step 14/32 : ARG GOOGLE_DRIVE_FOLDER_ID
 ---> Running in 65c5345933a4
Removing intermediate container 65c5345933a4
 ---> 9a38900bf9d4
Step 15/32 : ARG GOOGLE_CLOUD_NLP_CREDENTIALS
 ---> Running in 87a9c8e6ef4a
Removing intermediate container 87a9c8e6ef4a
 ---> fdd12cb65d9a
Step 16/32 : ARG GOOGLE_CLOUD_NLP_MODEL_NAME
 ---> Running in e2384898faa4
Removing intermediate container e2384898faa4
 ---> acbda2e77ea0
Step 17/32 : ENV DATABASE_HOSTNAME ${DATABASE_HOSTNAME}
 ---> Running in 3ef76a66a85d
Removing intermediate container 3ef76a66a85d
 ---> 70446d77d37e
Step 18/32 : ENV DATABASE_PASSWORD ${DATABASE_PASSWORD}
 ---> Running in 19cb15d63282
Removing intermediate container 19cb15d63282
 ---> 9057a33c4e94
Step 19/32 : ENV DATABASE_USERNAME ${DATABASE_USERNAME}
 ---> Running in 1492e273fa20
Removing intermediate container 1492e273fa20
 ---> 862726429857
Step 20/32 : ENV DATABASE_NAME ${DATABASE_NAME}
 ---> Running in 00b95ac5f7ea
Removing intermediate container 00b95ac5f7ea
 ---> 57e7fe2dfffa
Step 21/32 : ENV PYDRIVE_CLIENT_ID ${PYDRIVE_CLIENT_ID}
 ---> Running in d721d31eb624
Removing intermediate container d721d31eb624
 ---> a6600464716c
Step 22/32 : ENV PYDRIVE_CLIENT_SECRET ${PYDRIVE_CLIENT_SECRET}
 ---> Running in b197edc0ae83
Removing intermediate container b197edc0ae83
 ---> 6c8d4576bf12
Step 23/32 : ENV GOOGLE_DRIVE_CREDENTIALS ${GOOGLE_DRIVE_CREDENTIALS}
 ---> Running in 9df1f625a19c
Removing intermediate container 9df1f625a19c
 ---> 8d1f590f1f7a
Step 24/32 : ENV GOOGLE_DRIVE_FOLDER_ID ${GOOGLE_DRIVE_FOLDER_ID}
 ---> Running in 3ae756adfbff
Removing intermediate container 3ae756adfbff
 ---> 089ac4a00416
Step 25/32 : ENV GOOGLE_CLOUD_NLP_CREDENTIALS ${GOOGLE_CLOUD_NLP_CREDENTIALS}
 ---> Running in 3bb961d4ae8c
Removing intermediate container 3bb961d4ae8c
 ---> bd9e45764bae
Step 26/32 : ENV GOOGLE_CLOUD_NLP_MODEL_NAME ${GOOGLE_CLOUD_NLP_MODEL_NAME}
 ---> Running in efb294f0f567
Removing intermediate container efb294f0f567
 ---> 92185a606799
Step 27/32 : WORKDIR /nimbus
 ---> Running in c8288b4a31d4
Removing intermediate container c8288b4a31d4
 ---> 2d4894129e41
Step 28/32 : RUN ./setup_special_files_from_env.py
 ---> Running in 050398d53797
Removing intermediate container 050398d53797
 ---> f97224426ae8
Step 29/32 : RUN ./download_nlp_stuff.sh
 ---> Running in 55022b40f20b
Collecting en_core_web_sm==2.2.5
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.5/en_core_web_sm-2.2.5.tar.gz (12.0 MB)
Requirement already satisfied: spacy>=2.2.2 in /usr/local/lib/python3.6/dist-packages (from en_core_web_sm==2.2.5) (2.2.3)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.6/dist-packages (from spacy>=2.2.2->en_core_web_sm==2.2.5) (1.0.2)
Requirement already satisfied: wasabi<1.1.0,>=0.4.0 in /usr/local/lib/python3.6/dist-packages (from spacy>=2.2.2->en_core_web_sm==2.2.5) (0.6.0)
Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from spacy>=2.2.2->en_core_web_sm==2.2.5) (1.18.1)
Requirement already satisfied: catalogue<1.1.0,>=0.0.7 in /usr/local/lib/python3.6/dist-packages (from spacy>=2.2.2->en_core_web_sm==2.2.5) (1.0.0)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from spacy>=2.2.2->en_core_web_sm==2.2.5) (3.0.2)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3.6/dist-packages (from spacy>=2.2.2->en_core_web_sm==2.2.5) (2.23.0)
Requirement already satisfied: blis<0.5.0,>=0.4.0 in /usr/local/lib/python3.6/dist-packages (from spacy>=2.2.2->en_core_web_sm==2.2.5) (0.4.1)
Requirement already satisfied: plac<1.2.0,>=0.9.6 in /usr/local/lib/python3.6/dist-packages (from spacy>=2.2.2->en_core_web_sm==2.2.5) (1.1.3)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from spacy>=2.2.2->en_core_web_sm==2.2.5) (39.0.1)
Requirement already satisfied: srsly<1.1.0,>=0.1.0 in /usr/local/lib/python3.6/dist-packages (from spacy>=2.2.2->en_core_web_sm==2.2.5) (1.0.1)
Requirement already satisfied: thinc<7.4.0,>=7.3.0 in /usr/local/lib/python3.6/dist-packages (from spacy>=2.2.2->en_core_web_sm==2.2.5) (7.3.1)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.6/dist-packages (from spacy>=2.2.2->en_core_web_sm==2.2.5) (2.0.3)
Requirement already satisfied: importlib-metadata>=0.20; python_version < "3.8" in /usr/local/lib/python3.6/dist-packages (from catalogue<1.1.0,>=0.0.7->spacy>=2.2.2->en_core_web_sm==2.2.5) (1.5.0)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.2->en_core_web_sm==2.2.5) (2.9)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.2->en_core_web_sm==2.2.5) (1.25.8)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.2->en_core_web_sm==2.2.5) (2019.11.28)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests<3.0.0,>=2.13.0->spacy>=2.2.2->en_core_web_sm==2.2.5) (3.0.4)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /usr/local/lib/python3.6/dist-packages (from thinc<7.4.0,>=7.3.0->spacy>=2.2.2->en_core_web_sm==2.2.5) (4.43.0)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata>=0.20; python_version < "3.8"->catalogue<1.1.0,>=0.0.7->spacy>=2.2.2->en_core_web_sm==2.2.5) (3.0.0)
Building wheels for collected packages: en-core-web-sm
  Building wheel for en-core-web-sm (setup.py): started
  Building wheel for en-core-web-sm (setup.py): finished with status 'done'
  Created wheel for en-core-web-sm: filename=en_core_web_sm-2.2.5-py3-none-any.whl size=12012546 sha256=1eae991a1d7b9c047a3b0c498c672f572f6f02809e97d2fd937128b90e7ae775
  Stored in directory: /tmp/pip-ephem-wheel-cache-448snryk/wheels/b5/94/56/596daa677d7e91038cbddfcf32b591d0c915a1b3a3e3d3c79d
Successfully built en-core-web-sm
Installing collected packages: en-core-web-sm
Successfully installed en-core-web-sm-2.2.5
[+] Download and installation successful
You can now load the model via spacy.load('en_core_web_sm')
[nltk_data] Downloading package punkt to /root/nltk_data...
[nltk_data]   Unzipping tokenizers/punkt.zip.
[nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data]     /root/nltk_data...
[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.
Removing intermediate container 55022b40f20b
 ---> e4c52eab5707
Step 30/32 : RUN ls | grep config
 ---> Running in 1286c66e3acd
config.json
config_SAMPLE.json
gunicorn_config.py
Removing intermediate container 1286c66e3acd
 ---> ff53b3f4f97f
Step 31/32 : WORKDIR /nimbus
 ---> Running in f50a77b8dac1
Removing intermediate container f50a77b8dac1
 ---> 7447eaa2a2fb
Step 32/32 : CMD ["gunicorn", "flask_api:app", "--config=gunicorn_config.py"]
 ---> Running in 298b2934659f
Removing intermediate container 298b2934659f
 ---> f3cab2394c52
Successfully built f3cab2394c52
Successfully tagged mfekadu/nimbus:latest

docker run

$ docker run -it --rm -p 8080:8080  mfekadu/nimbus

[2020-03-02 00:15:10 +0000] [1] [INFO] Starting gunicorn 20.0.4
[2020-03-02 00:15:10 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2020-03-02 00:15:10 +0000] [1] [INFO] Using worker: sync
[2020-03-02 00:15:10 +0000] [9] [INFO] Booting worker with pid: 9
[2020-03-02 00:15:10 +0000] [10] [INFO] Booting worker with pid: 10
[2020-03-02 00:15:10 +0000] [11] [INFO] Booting worker with pid: 11
initialized database session
initialized NimbusMySQLAlchemy
initialized database session
initialized NimbusMySQLAlchemy
initialized database session
initialized NimbusMySQLAlchemy
[nltk_data] Downloading package stopwords to /root/nltk_data...
[nltk_data]   Unzipping corpora/stopwords.zip.
/usr/local/lib/python3.6/dist-packages/sklearn/base.py:251: UserWarning: Trying to unpickle estimator KNeighborsClassifier from version 0.21.3 when using version 0.20.2. This might lead to breaking code or invalid results. Use at your own risk.
  UserWarning)
{'entity': 'Dr. Allen', 'tag': 'PROF', 'normalized entity': 'Allen', 'input question': "What is Dr. Allen's phone number?", 'normalized question': "What is [PROF]'s phone number?", 'question class': "What is [PROF]'s phone number?"}
mfekadu commented 4 years ago

it's weird to still see the nimbus seg fault @snekiam

$ docker run -it --rm mfekadu/nimbus sh

# cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
# uname -r
4.19.76-linuxkit
# python --version
Python 3.6.9
# python nimbus.py
Segmentation fault
# ^[