api

GitHub repo size GitHub code size in bytes GitHub closed issues GitHub closed pull requests

Official API for the NIMBUS Voice Assistant accessible via HTTP REST protocol.

🚧 This API is still in-development, so expect the endpoints to be constantly changing until finalized.

GitHub Actions Status

Documentation

[ ] TODO: INSERT LINK TO API DOCUMENTATION
[ ] TODO: USE http://readthedocs.org to host the docs live on the internet
[ ] TODO: CREATE API DOCS 😅

Dev Environment Setup

Prerequisites

Python 3.6.9+ (we're running 3.8 on gce)
pip 9.0.1+ (pip 20.1.1 is whats on gce as of this writing, but whatever version you have with your python install is probably fine)
git (can you run git --version?) (we're using 2.25.1 on gce, but any version will almost certainly work)
Access to database credentials/api keys (contact us)

If you're running Windows, we recommend (it is not required) setting up the Windows subsystem for Linux.

Database Configuration

There are THREE ways to get your database config setup. If you're a member of CSAI, there's a script to generate a config file - contact a club member to get access to it.

Create a file called config.json that should include at least the following details of a MySQL database:

{
    ...
    "mysql": {
        "host": "HOSTNAME",
        "port": "PORT e.g. 3306",
        "user": "USERNAME",
        "password": "PASSWORD",
        "database": "DATABASE",
        ...
    }
    ...
}

You can also use config_SAMPLE.json as a reference (easier!)

Contact anyone on the Data Team to get connection details for the Nimbus database

Install pip packages

Note: If you don't have pip installed for python2, you may have to call pip instead of pip3)

pip3 install -r requirements.txt

Download spaCy model

There are two spaCy models usable for nimbus predictions. These are (from spaCy's docs) General-purpose pretrained models to predict named entities, part-of-speech tags and syntactic dependencies. Can be used out-of-the-box and fine-tuned on more specific data. We use these to classify a question someone asks Nimbus, so we can retrieve an answer from our database.See here for an idea of the difference between lg and sm.

python3 -m spacy download en_core_web_sm

python3 -m spacy download en_core_web_lg

Get API keys for terminal instance

git clone https://github.com/calpoly-csai/nimbus-config-secrets.git

cd nimbus-config-secrets 

source .export_env_vars

You should now have everything you need to run the API server.

Run the API server

python3 flask_api.py

Opens localhost:PORT
PORT is changed in gunicorn_config.py

Run in the background

python3 flask_api.py&

Run your first API request

curl --location --request POST 'localhost:8080/ask' --header 'Content-Type: application/json' --data-raw '{ "question": "What are the prereqs for CSC 480?" }'

Sends Query to Database through localhost:8080/ask

Run the tests

pytest

(you may have to run the following if you have python 2.7 installed as well):

python3 -m pytest

Python PEP8 Style Standards

Run the format script to automatically make our code look nice

./format.sh

Sometimes the format script is not enough, so run lint to manually style our code

./lint.sh

Some common issues and solutions

`/bin/bash^M: bad interpreter: No such file or directory`:

Note: This issue can occur (for me, at least) in files other than just format.sh. These instructions should fix this problem in any file.
snekiam@P1:api$ ./format.sh
-bash: ./format.sh: /bin/bash^M: bad interpreter: No such file or directory
This can happen when you're running bash on windows sometimes. Linux doesn't recognize carriage returns (^M) the same way windows does. You may have this error in more than one file, but it can be fixed by running dos2unix like this:
snekiam@P1:api$ dos2unix format.sh
dos2unix: converting file format.sh to Unix format...
snekiam@P1:api$ ./format.sh
(output of format.sh, which can be long so I haven't included it here)

Deployment

What we use

A Linux server (e.g. Ubuntu 20.04 LTS) with open firewall at tcp:5000 for Flask, tcp:80 for http and tcp:443 for https and tcp:22 for ssh and tcp:3306 for mysql

See this documentation of the database deployment process

We're using docker to deploy - at some point, it'll be easy for you to run the Nimbus API in a local docker container, but at the moment it requires some things (like SSH keys and SSL private keys) which shouldn't be required for a development environment.