search

bananaml / potassium

An HTTP serving framework by Banana
Apache License 2.0
97 stars 9 forks source link

POTASSIUM_ERROR NameError: name 'DBAPIConnection' is not defined #23

Open Rams901 opened 1 year ago

Rams901 commented 1 year ago

When deploying a model and still following the exact documentation for migrating a model to banana v2. When using potassium, I have encountered the following error in the health check; (Passed deployment) Last part of the build logs: `You've triggered a build + deploy on Banana. It may take 5-15 min to complete. Thanks for your patience. Waiting for logs...

SUCCESS: Git Authorization SUCCESS: Build Started Step 1/11 : FROM pytorch/pytorch:1.11.0-cuda11.3-cudnn8-runtime ---> ca04e7f7c8e5

Status: Image is up to date for pytorch/pytorch:1.11.0-cuda11.3-cudnn8-runtime
Pulling from pytorch/pytorch 1.11.0-cuda11.3-cudnn8-runtimeStep 2/11 : WORKDIR / ---> Using cache ---> 786d6ef93c68 Step 3/11 : RUN apt-get update && apt-get install -y git ---> Using cache ---> e9c7bf12d357 Step 4/11 : RUN pip3 install --upgrade pip ---> Using cache ---> 51e860ef003f Step 5/11 : ADD requirements.txt requirements.txt ---> 20108ea617d4 Step 6/11 : RUN pip3 install -r requirements.txt ---> Running in e52d25d6a8f4 Collecting potassium (from -r requirements.txt (line 1)) Downloading potassium-0.1.1-py3-none-any.whl (12 kB) Requirement already satisfied: torch in /opt/conda/lib/python3.8/site-packages (from -r requirements.txt (line 2)) (1.11.0) Collecting transformers (from -r requirements.txt (line 3)) Downloading transformers-4.30.2-py3-none-any.whl (7.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.2/7.2 MB 102.1 MB/s eta 0:00:00 Collecting langchain (from -r requirements.txt (line 4)) Downloading langchain-0.0.223-py3-none-any.whl (1.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 116.7 MB/s eta 0:00:00 Collecting einops (from -r requirements.txt (line 5)) Downloading einops-0.6.1-py3-none-any.whl (42 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.2/42.2 kB 14.7 MB/s eta 0:00:00Collecting accelerate (from -r requirements.txt (line 6)) Downloading accelerate-0.20.3-py3-none-any.whl (227 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 227.6/227.6 kB 50.1 MB/s eta 0:00:00Collecting xformers (from -r requirements.txt (line 7)) Downloading xformers-0.0.20-cp38-cp38-manylinux2014_x86_64.whl (109.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 109.1/109.1 MB 27.4 MB/s eta 0:00:00Collecting Flask (from potassium->-r requirements.txt (line 1)) Downloading Flask-2.3.2-py3-none-any.whl (96 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.9/96.9 kB 26.8 MB/s eta 0:00:00Requirement already satisfied: requests in /opt/conda/lib/python3.8/site-packages (from potassium->-r requirements.txt (line 1)) (2.27.1) Collecting termcolor (from potassium->-r requirements.txt (line 1)) Downloading termcolor-2.3.0-py3-none-any.whl (6.9 kB) Collecting redis (from potassium->-r requirements.txt (line 1)) Downloading redis-4.6.0-py3-none-any.whl (241 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 241.1/241.1 kB 50.3 MB/s eta 0:00:00Collecting boto3 (from potassium->-r requirements.txt (line 1)) Downloading boto3-1.27.0-py3-none-any.whl (135 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 135.9/135.9 kB 36.8 MB/s eta 0:00:00Requirement already satisfied: typing_extensions in /opt/conda/lib/python3.8/site-packages (from torch->-r requirements.txt (line 2)) (3.10.0.2) Requirement already satisfied: filelock in /opt/conda/lib/python3.8/site-packages (from transformers->-r requirements.txt (line 3)) (3.4.2) Collecting huggingface-hub<1.0,>=0.14.1 (from transformers->-r requirements.txt (line 3)) Downloading huggingface_hub-0.16.2-py3-none-any.whl (268 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 268.5/268.5 kB 66.2 MB/s eta 0:00:00Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.8/site-packages (from transformers->-r requirements.txt (line 3)) (1.21.2) Collecting packaging>=20.0 (from transformers->-r requirements.txt (line 3)) Downloading packaging-23.1-py3-none-any.whl (48 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 13.2 MB/s eta 0:00:00Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.8/site-packages (from transformers->-r requirements.txt (line 3)) (6.0) Collecting regex!=2019.12.17 (from transformers->-r requirements.txt (line 3)) Downloading regex-2023.6.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (772 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 772.3/772.3 kB 102.5 MB/s eta 0:00:00Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers->-r requirements.txt (line 3)) Downloading tokenizers-0.13.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 79.4 MB/s eta 0:00:00 Collecting safetensors>=0.3.1 (from transformers->-r requirements.txt (line 3)) Downloading safetensors-0.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 112.4 MB/s eta 0:00:00 Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.8/site-packages (from transformers->-r requirements.txt (line 3)) (4.62.3) Collecting SQLAlchemy<3,>=1.4 (from langchain->-r requirements.txt (line 4)) Downloading SQLAlchemy-2.0.17-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.8/2.8 MB 125.6 MB/s eta 0:00:00Collecting aiohttp<4.0.0,>=3.8.3 (from langchain->-r requirements.txt (line 4)) Downloading aiohttp-3.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 121.7 MB/s eta 0:00:00 Collecting async-timeout<5.0.0,>=4.0.0 (from langchain->-r requirements.txt (line 4)) Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB) Collecting dataclasses-json<0.6.0,>=0.5.7 (from langchain->-r requirements.txt (line 4)) Downloading dataclasses_json-0.5.9-py3-none-any.whl (26 kB) Collecting langchainplus-sdk<0.0.21,>=0.0.20 (from langchain->-r requirements.txt (line 4)) Downloading langchainplus_sdk-0.0.20-py3-none-any.whl (25 kB) Collecting numexpr<3.0.0,>=2.8.4 (from langchain->-r requirements.txt (line 4)) Downloading numexpr-2.8.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (381 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 381.7/381.7 kB 82.2 MB/s eta 0:00:00Collecting openapi-schema-pydantic<2.0,>=1.2 (from langchain->-r requirements.txt (line 4)) Downloading openapi_schema_pydantic-1.2.4-py3-none-any.whl (90 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.0/90.0 kB 31.9 MB/s eta 0:00:00Collecting pydantic<2,>=1 (from langchain->-r requirements.txt (line 4)) Downloading pydantic-1.10.11-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 143.3 MB/s eta 0:00:00 Collecting tenacity<9.0.0,>=8.1.0 (from langchain->-r requirements.txt (line 4)) Downloading tenacity-8.2.2-py3-none-any.whl (24 kB) Requirement already satisfied: psutil in /opt/conda/lib/python3.8/site-packages (from accelerate->-r requirements.txt (line 6)) (5.8.0) Collecting pyre-extensions==0.0.29 (from xformers->-r requirements.txt (line 7)) Downloading pyre_extensions-0.0.29-py3-none-any.whl (12 kB) Collecting torch (from -r requirements.txt (line 2)) Downloading torch-2.0.1-cp38-cp38-manylinux1_x86_64.whl (619.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 619.9/619.9 MB 6.9 MB/s eta 0:00:00 Collecting sympy (from torch->-r requirements.txt (line 2)) Downloading sympy-1.12-py3-none-any.whl (5.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 103.7 MB/s eta 0:00:00Collecting networkx (from torch->-r requirements.txt (line 2)) Downloading networkx-3.1-py3-none-any.whl (2.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 136.5 MB/s eta 0:00:00Requirement already satisfied: jinja2 in /opt/conda/lib/python3.8/site-packages (from torch->-r requirements.txt (line 2)) (2.11.3) Collecting nvidia-cuda-nvrtc-cu11==11.7.99 (from torch->-r requirements.txt (line 2)) Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.0/21.0 MB 85.9 MB/s eta 0:00:00 Collecting nvidia-cuda-runtime-cu11==11.7.99 (from torch->-r requirements.txt (line 2)) Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 849.3/849.3 kB 106.2 MB/s eta 0:00:00Collecting nvidia-cuda-cupti-cu11==11.7.101 (from torch->-r requirements.txt (line 2)) Downloading nvidia_cuda_cupti_cu11-11.7.101-py3-none-manylinux1_x86_64.whl (11.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.8/11.8 MB 127.1 MB/s eta 0:00:00Collecting nvidia-cudnn-cu11==8.5.0.96 (from torch->-r requirements.txt (line 2)) Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 557.1/557.1 MB 4.9 MB/s eta 0:00:00 Collecting nvidia-cublas-cu11==11.10.3.66 (from torch->-r requirements.txt (line 2)) Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 317.1/317.1 MB 8.8 MB/s eta 0:00:00 Collecting nvidia-cufft-cu11==10.9.0.58 (from torch->-r requirements.txt (line 2)) Downloading nvidia_cufft_cu11-10.9.0.58-py3-none-manylinux1_x86_64.whl (168.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 168.4/168.4 MB 21.2 MB/s eta 0:00:00Collecting nvidia-curand-cu11==10.2.10.91 (from torch->-r requirements.txt (line 2)) Downloading nvidia_curand_cu11-10.2.10.91-py3-none-manylinux1_x86_64.whl (54.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.6/54.6 MB 52.8 MB/s eta 0:00:00Collecting nvidia-cusolver-cu11==11.4.0.1 (from torch->-r requirements.txt (line 2)) Downloading nvidia_cusolver_cu11-11.4.0.1-2-py3-none-manylinux1_x86_64.whl (102.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.6/102.6 MB 30.3 MB/s eta 0:00:00Collecting nvidia-cusparse-cu11==11.7.4.91 (from torch->-r requirements.txt (line 2)) Downloading nvidia_cusparse_cu11-11.7.4.91-py3-none-manylinux1_x86_64.whl (173.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 173.2/173.2 MB 20.3 MB/s eta 0:00:00 Collecting nvidia-nccl-cu11==2.14.3 (from torch->-r requirements.txt (line 2)) Downloading nvidia_nccl_cu11-2.14.3-py3-none-manylinux1_x86_64.whl (177.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 177.1/177.1 MB 22.2 MB/s eta 0:00:00 Collecting nvidia-nvtx-cu11==11.7.91 (from torch->-r requirements.txt (line 2)) Downloading nvidia_nvtx_cu11-11.7.91-py3-none-manylinux1_x86_64.whl (98 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.6/98.6 kB 29.5 MB/s eta 0:00:00Collecting triton==2.0.0 (from torch->-r requirements.txt (line 2)) Downloading triton-2.0.0-1-cp38-cp38-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (63.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.2/63.2 MB 45.3 MB/s eta 0:00:00Requirement already satisfied: setuptools in /opt/conda/lib/python3.8/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch->-r requirements.txt (line 2)) (58.0.4) Requirement already satisfied: wheel in /opt/conda/lib/python3.8/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch->-r requirements.txt (line 2)) (0.37.1) Collecting typing-inspect (from pyre-extensions==0.0.29->xformers->-r requirements.txt (line 7)) Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB) Collecting cmake (from triton==2.0.0->torch->-r requirements.txt (line 2)) Downloading cmake-3.26.4-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (24.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 24.0/24.0 MB 84.9 MB/s eta 0:00:00Collecting lit (from triton==2.0.0->torch->-r requirements.txt (line 2)) Downloading lit-16.0.6.tar.gz (153 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 153.7/153.7 kB 48.9 MB/s eta 0:00:00 Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Installing backend dependencies: started Installing backend dependencies: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting attrs>=17.3.0 (from aiohttp<4.0.0,>=3.8.3->langchain->-r requirements.txt (line 4)) Downloading attrs-23.1.0-py3-none-any.whl (61 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 14.3 MB/s eta 0:00:00Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /opt/conda/lib/python3.8/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain->-r requirements.txt (line 4)) (2.0.4) Collecting multidict<7.0,>=4.5 (from aiohttp<4.0.0,>=3.8.3->langchain->-r requirements.txt (line 4)) Downloading multidict-6.0.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (121 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.3/121.3 kB 38.7 MB/s eta 0:00:00Collecting yarl<2.0,>=1.0 (from aiohttp<4.0.0,>=3.8.3->langchain->-r requirements.txt (line 4)) Downloading yarl-1.9.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (266 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 266.9/266.9 kB 69.8 MB/s eta 0:00:00Collecting frozenlist>=1.1.1 (from aiohttp<4.0.0,>=3.8.3->langchain->-r requirements.txt (line 4)) Downloading frozenlist-1.3.3-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (161 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 161.3/161.3 kB 34.3 MB/s eta 0:00:00Collecting aiosignal>=1.1.2 (from aiohttp<4.0.0,>=3.8.3->langchain->-r requirements.txt (line 4)) Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB) Collecting marshmallow<4.0.0,>=3.3.0 (from dataclasses-json<0.6.0,>=0.5.7->langchain->-r requirements.txt (line 4)) Downloading marshmallow-3.19.0-py3-none-any.whl (49 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.1/49.1 kB 14.1 MB/s eta 0:00:00Collecting marshmallow-enum<2.0.0,>=1.5.1 (from dataclasses-json<0.6.0,>=0.5.7->langchain->-r requirements.txt (line 4)) Downloading marshmallow_enum-1.5.1-py2.py3-none-any.whl (4.2 kB) Collecting fsspec (from huggingface-hub<1.0,>=0.14.1->transformers->-r requirements.txt (line 3)) Downloading fsspec-2023.6.0-py3-none-any.whl (163 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 163.8/163.8 kB 39.9 MB/s eta 0:00:00 Collecting typing-extensions (from torch->-r requirements.txt (line 2)) Downloading typing_extensions-4.7.1-py3-none-any.whl (33 kB) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.8/site-packages (from requests->potassium->-r requirements.txt (line 1)) (1.26.7) Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.8/site-packages (from requests->potassium->-r requirements.txt (line 1)) (2021.10.8) Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.8/site-packages (from requests->potassium->-r requirements.txt (line 1)) (3.3) Collecting greenlet!=0.4.17 (from SQLAlchemy<3,>=1.4->langchain->-r requirements.txt (line 4)) Downloading greenlet-2.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (618 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 618.5/618.5 kB 88.5 MB/s eta 0:00:00Collecting botocore<1.31.0,>=1.30.0 (from boto3->potassium->-r requirements.txt (line 1)) Downloading botocore-1.30.0-py3-none-any.whl (11.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.0/11.0 MB 132.6 MB/s eta 0:00:00 Collecting jmespath<2.0.0,>=0.7.1 (from boto3->potassium->-r requirements.txt (line 1)) Downloading jmespath-1.0.1-py3-none-any.whl (20 kB) Collecting s3transfer<0.7.0,>=0.6.0 (from boto3->potassium->-r requirements.txt (line 1)) Downloading s3transfer-0.6.1-py3-none-any.whl (79 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.8/79.8 kB 19.2 MB/s eta 0:00:00Collecting Werkzeug>=2.3.3 (from Flask->potassium->-r requirements.txt (line 1)) Downloading Werkzeug-2.3.6-py3-none-any.whl (242 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 242.5/242.5 kB 57.9 MB/s eta 0:00:00Collecting jinja2 (from torch->-r requirements.txt (line 2)) Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 38.7 MB/s eta 0:00:00Collecting itsdangerous>=2.1.2 (from Flask->potassium->-r requirements.txt (line 1)) Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB) Collecting click>=8.1.3 (from Flask->potassium->-r requirements.txt (line 1)) Downloading click-8.1.3-py3-none-any.whl (96 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 29.0 MB/s eta 0:00:00Collecting blinker>=1.6.2 (from Flask->potassium->-r requirements.txt (line 1)) Downloading blinker-1.6.2-py3-none-any.whl (13 kB) Collecting importlib-metadata>=3.6.0 (from Flask->potassium->-r requirements.txt (line 1)) Downloading importlib_metadata-6.7.0-py3-none-any.whl (22 kB) Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.8/site-packages (from jinja2->torch->-r requirements.txt (line 2)) (2.0.1) Collecting mpmath>=0.19 (from sympy->torch->-r requirements.txt (line 2)) Downloading mpmath-1.3.0-py3-none-any.whl (536 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 92.7 MB/s eta 0:00:00Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.31.0,>=1.30.0->boto3->potassium->-r requirements.txt (line 1)) Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 56.2 MB/s eta 0:00:00Collecting zipp>=0.5 (from importlib-metadata>=3.6.0->Flask->potassium->-r requirements.txt (line 1)) Downloading zipp-3.15.0-py3-none-any.whl (6.8 kB) Collecting mypy-extensions>=0.3.0 (from typing-inspect->pyre-extensions==0.0.29->xformers->-r requirements.txt (line 7)) Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB) Collecting MarkupSafe>=2.0 (from jinja2->torch->-r requirements.txt (line 2)) Downloading MarkupSafe-2.1.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB) Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.8/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.31.0,>=1.30.0->boto3->potassium->-r requirements.txt (line 1)) (1.16.0) Building wheels for collected packages: lit Building wheel for lit (pyproject.toml): started Building wheel for lit (pyproject.toml): finished with status 'done' Created wheel for lit: filename=lit-16.0.6-py3-none-any.whl size=93582 sha256=7c912b8cf3a764ad0c4f629ea228753ebd9a43184b4af02b5951e74cd2b1ae2f Stored in directory: /root/.cache/pip/wheels/05/ab/f1/0102fea49a41c753f0e79a1a4012417d5d7ef0f93224694472 Successfully built lit Installing collected packages: tokenizers, safetensors, mpmath, lit, cmake, zipp, typing-extensions, termcolor, tenacity, sympy, regex, python-dateutil, packaging, nvidia-nvtx-cu11, nvidia-nccl-cu11, nvidia-cusparse-cu11, nvidia-curand-cu11, nvidia-cufft-cu11, nvidia-cuda-runtime-cu11, nvidia-cuda-nvrtc-cu11, nvidia-cuda-cupti-cu11, nvidia-cublas-cu11, numexpr, networkx, mypy-extensions, multidict, MarkupSafe, jmespath, itsdangerous, greenlet, fsspec, frozenlist, einops, click, blinker, attrs, async-timeout, yarl, Werkzeug, typing-inspect, SQLAlchemy, redis, pydantic, nvidia-cusolver-cu11, nvidia-cudnn-cu11, marshmallow, jinja2, importlib-metadata, huggingface-hub, botocore, aiosignal, transformers, s3transfer, pyre-extensions, openapi-schema-pydantic, marshmallow-enum, langchainplus-sdk, Flask, aiohttp, dataclasses-json, boto3, potassium, langchain, triton, torch, xformers, accelerate Attempting uninstall: typing-extensions Found existing installation: typing-extensions 3.10.0.2 Uninstalling typing-extensions-3.10.0.2: Successfully uninstalled typing-extensions-3.10.0.2 Attempting uninstall: MarkupSafe Found existing installation: MarkupSafe 2.0.1 Uninstalling MarkupSafe-2.0.1: Successfully uninstalled MarkupSafe-2.0.1 Attempting uninstall: jinja2 Found existing installation: Jinja2 2.11.3 Uninstalling Jinja2-2.11.3: Successfully uninstalled Jinja2-2.11.3 Attempting uninstall: torch Found existing installation: torch 1.11.0 Uninstalling torch-1.11.0: Successfully uninstalled torch-1.11.0 Successfully installed Flask-2.3.2 MarkupSafe-2.1.3 SQLAlchemy-2.0.17 Werkzeug-2.3.6 accelerate-0.20.3 aiohttp-3.8.4 aiosignal-1.3.1 async-timeout-4.0.2 attrs-23.1.0 blinker-1.6.2 boto3-1.27.0 botocore-1.30.0 click-8.1.3 cmake-3.26.4 dataclasses-json-0.5.9 einops-0.6.1 frozenlist-1.3.3 fsspec-2023.6.0 greenlet-2.0.2 huggingface-hub-0.16.2 importlib-metadata-6.7.0 itsdangerous-2.1.2 jinja2-3.1.2 jmespath-1.0.1 langchain-0.0.223 langchainplus-sdk-0.0.20 lit-16.0.6 marshmallow-3.19.0 marshmallow-enum-1.5.1 mpmath-1.3.0 multidict-6.0.4 mypy-extensions-1.0.0 networkx-3.1 numexpr-2.8.4 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-cupti-cu11-11.7.101 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.2.10.91 nvidia-cusolver-cu11-11.4.0.1 nvidia-cusparse-cu11-11.7.4.91 nvidia-nccl-cu11-2.14.3 nvidia-nvtx-cu11-11.7.91 openapi-schema-pydantic-1.2.4 packaging-23.1 potassium-0.1.1 pydantic-1.10.11 pyre-extensions-0.0.29 python-dateutil-2.8.2 redis-4.6.0 regex-2023.6.3 s3transfer-0.6.1 safetensors-0.3.1 sympy-1.12 tenacity-8.2.2 termcolor-2.3.0 tokenizers-0.13.3 torch-2.0.1 transformers-4.30.2 triton-2.0.0 typing-extensions-4.7.1 typing-inspect-0.9.0 xformers-0.0.20 yarl-1.9.2 zipp-3.15.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv ---> e58fab5bd588 Step 7/11 : ADD download.py . ---> 25875abb06c3 Step 8/11 : RUN python3 download.py ---> Running in 46efc86b34be  Downloading (…)okenizer_config.json: 0%| | 0.00/220 [00:00<?, ?B/s] Downloading (…)okenizer_config.json: 100%|██████████| 220/220 [00:00<00:00, 53.3kB/s]

Downloading (…)/main/tokenizer.json: 0%| | 0.00/2.73M [00:00<?, ?B/s] Downloading (…)/main/tokenizer.json: 100%|██████████| 2.73M/2.73M [00:00<00:00, 21.3MB/s] Downloading (…)/main/tokenizer.json: 100%|██████████| 2.73M/2.73M [00:00<00:00, 21.1MB/s]

Downloading (…)cial_tokens_map.json: 0%| | 0.00/281 [00:00<?, ?B/s] Downloading (…)cial_tokens_map.json: 100%|██████████| 281/281 [00:00<00:00, 532kB/s]

Downloading (…)lve/main/config.json: 0%| | 0.00/667 [00:00<?, ?B/s] Downloading (…)lve/main/config.json: 100%|██████████| 667/667 [00:00<00:00, 171kB/s]

Downloading (…)/configuration_RW.py: 0%| | 0.00/2.61k [00:00<?, ?B/s] Downloading (…)/configuration_RW.py: 100%|██████████| 2.61k/2.61k [00:00<00:00, 4.15MB/s] A new version of the following files was downloaded from https://huggingface.co/tiiuae/falcon-7b-instruct:

Loading checkpoint shards: 100%|██████████| 2/2 [00:07<00:00, 3.99s/it]

Downloading (…)neration_config.json: 0%| | 0.00/111 [00:00<?, ?B/s] Downloading (…)neration_config.json: 100%|██████████| 111/111 [00:00<00:00, 34.5kB/s] The model 'RWForCausalLM' is not supported for text-generation. Supported models are ['BartForCausalLM', 'BertLMHeadModel', 'BertGenerationDecoder', 'BigBirdForCausalLM', 'BigBirdPegasusForCausalLM', 'BioGptForCausalLM', 'BlenderbotForCausalLM', 'BlenderbotSmallForCausalLM', 'BloomForCausalLM', 'CamembertForCausalLM', 'CodeGenForCausalLM', 'CpmAntForCausalLM', 'CTRLLMHeadModel', 'Data2VecTextForCausalLM', 'ElectraForCausalLM', 'ErnieForCausalLM', 'GitForCausalLM', 'GPT2LMHeadModel', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTNeoForCausalLM', 'GPTNeoXForCausalLM', 'GPTNeoXJapaneseForCausalLM', 'GPTJForCausalLM', 'LlamaForCausalLM', 'MarianForCausalLM', 'MBartForCausalLM', 'MegaForCausalLM', 'MegatronBertForCausalLM', 'MvpForCausalLM', 'OpenLlamaForCausalLM', 'OpenAIGPTLMHeadModel', 'OPTForCausalLM', 'PegasusForCausalLM', 'PLBartForCausalLM', 'ProphetNetForCausalLM', 'QDQBertLMHeadModel', 'ReformerModelWithLMHead', 'RemBertForCausalLM', 'RobertaForCausalLM', 'RobertaPreLayerNormForCausalLM', 'RoCBertForCausalLM', 'RoFormerForCausalLM', 'RwkvForCausalLM', 'Speech2Text2ForCausalLM', 'TransfoXLLMHeadModel', 'TrOCRForCausalLM', 'XGLMForCausalLM', 'XLMWithLMHeadModel', 'XLMProphetNetForCausalLM', 'XLMRobertaForCausalLM', 'XLMRobertaXLForCausalLM', 'XLNetLMHeadModel', 'XmodForCausalLM']. ---> c753c1b3ea30 Step 9/11 : ADD . . ---> 767e2f5e1b16 Step 10/11 : EXPOSE 8000 ---> Running in 969ddb9c5009 ---> 138e5245fae6 Step 11/11 : CMD python3 -u app.py ---> Running in 220a5b12a443 ---> 1335f5d336ec Successfully built 1335f5d336ec Successfully tagged bananaml/cicero-ly_falcon-7b-summary-serverless-inference_3f5e1c94-489b-4bd8-bdd9-2d9135b7e263:0.0.1 Step 1/5 : ARG imagenameStep 2/5 : FROM ${imagename} ---> 1335f5d336ec Step 3/5 : ADD 🍌.py 🍌.py ---> 70fc38aaeed0 Step 4/5 : RUN python3 🍌.py ---> Running in 7e909ab18e6b Checking Potassium requirements... Checking installed version of potassium Detected installed version: 0.1.1 ---> 43857c622824 Step 5/5 : RUN rm 🍌.py ---> Running in db63e4aabde1 ---> ce34df273a6b Successfully built ce34df273a6b Successfully tagged bananaml/cicero-ly_falcon-7b-summary-serverless-inference_3f5e1c94-489b-4bd8-bdd9-2d9135b7e263:0.0.1 ERROR: Encountered an error the second healthcheck COVER BUILD_MODE = true COVER SERVER STARTED... now waiting for action! COVER HANDLER <-- Request received, method: POST, path: /turbo-init COVER torch cuda device count is: 1 COVER server is not ready yet POTASSIUM_ERROR NameError: name 'DBAPIConnection' is not defined COVER user code has unrecoverable error, shutting down everything

INTERNAL ERROR: Build server encountered an unexpected error`

sanjay-nit commented 1 year ago

I'm also getting the same Error.

erik-dunteman commented 1 year ago

Hi! Looks like the downstream effect of an import error, as the base healthcheck (your potassium server, unmodified) passes, meaning your code should be valid. When we build for Banana we use a custom python runtime, so chances are the pip3 environment is goofed (on our end).

To help us debug, could you share how you're importing your packages from app.py?

erik-dunteman commented 1 year ago

and @sanjay-nit are you too seeing this issue when running falcon?

Rams901 commented 1 year ago 
from potassium import Potassium, Request, Response

import transformers
import torch
from langchain.llms import HuggingFacePipeline
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains.summarize import load_summarize_chain
import sys

Hi @erik-dunteman, these are the imported packages in app.py

Hi! Looks like the downstream effect of an import error, as the base healthcheck (your potassium server, unmodified) passes, meaning your code should be valid. When we build for Banana we use a custom python runtime, so chances are the pip3 environment is goofed (on our end).

To help us debug, could you share how you're importing your packages from app.py?

erik-dunteman commented 1 year ago

Thanks @Rams901 and sorry for leaving this issue unanswered for so long. Let's get to the bottom of this

app.py imports look sane, though I don't know the ins-and-outs of the langchain package so couldn't ensure it's not an issue on their end. Perhaps they run conditional imports under the hood that'd lead to this DBAPIConnection name not being found.

Other things on Banana's end we can help debug:

Rams901 commented 1 year ago

@erik-dunteman, No problem, we just need to detect the problem location so we can advance and find solutions. I've done testing on colab since I don't have enough GPU Ram to test on my computer. But, I didn't test using docker. Also the model used takes around 15GB of GPU Ram is that okay given that the build + healthcheck machines run 23GB?

sanjay-nit commented 1 year ago

@erik-dunteman @Rams901 Any solutions for the above problems? I'm still facing this problem with langchain and kor. Is this problem specific to python 3.8 or something else? Because I'm able to run small LLM models in colab in python 3.10.6

erik-dunteman commented 1 year ago

Hi everyone, this one was a beast but hopefully we've got it solved. We have a PR in review, and will update once merged

sanjay-nit commented 1 year ago

Hi everyone, this one was a beast but hopefully we've got it solved. We have a PR in review, and will update once merged

@erik-dunteman langchain is working fine now. Thanks!

Here is one more follow up: Can we configure multiple banana GPUs for running Large Language Models like Llama2_70B?