search

lucataco / serverless-template-flan-t5

Basic template for using Flan-t5 on Banana's serverless GPU platform. Ready for 1-Click deploy
MIT License
10 stars 22 forks source link

Failed deployment for t5-xl #1

Closed gandalf-repo closed 1 year ago

gandalf-repo commented 1 year ago

I was following this article: https://www.banana.dev/blog/how-to-deploy-flan-t5-to-production-serverless-gpu After I tried to deploy it, it failed and shows status: Requirements Not Met

Here are the logs

2023-08-13T17:58:24.000Z You've triggered a build + deploy on Banana. It may take 5-15 min to complete. Thanks for your patience. 
Waiting for logs...

SUCCESS: Git Authorization
SUCCESS: Build Started
Step 1/12 : FROM pytorch/pytorch:1.11.0-cuda11.3-cudnn8-runtime

 ---> ca04e7f7c8e5

Status: Image is up to date for pytorch/pytorch:1.11.0-cuda11.3-cudnn8-runtime  
Pulling from pytorch/pytorch  1.11.0-cuda11.3-cudnn8-runtimeStep 2/12 : WORKDIR /

 ---> Using cache

 ---> 786d6ef93c68

Step 3/12 : RUN apt-get update && apt-get install -y git

 ---> Using cache

 ---> e9c7bf12d357

Step 4/12 : RUN pip3 install --upgrade pip

 ---> Using cache

 ---> 51e860ef003f

Step 5/12 : ADD requirements.txt requirements.txt

 ---> cc1a8eab5b04

Step 6/12 : RUN pip3 install -r requirements.txt

 ---> Running in a3ebf73b67e1

Collecting sanic==22.6.2 (from -r requirements.txt (line 1))

  Downloading sanic-22.6.2-py3-none-any.whl (271 kB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 271.9/271.9 kB 36.3 MB/s eta 0:00:00Collecting transformers==4.24.0 (from -r requirements.txt (line 2))

  Downloading transformers-4.24.0-py3-none-any.whl (5.5 MB)

Collecting accelerate==0.14.0 (from -r requirements.txt (line 3))

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.5/5.5 MB 89.2 MB/s eta 0:00:00
  Downloading accelerate-0.14.0-py3-none-any.whl (175 kB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 176.0/176.0 kB 51.3 MB/s eta 0:00:00Collecting sentencepiece==0.1.97 (from -r requirements.txt (line 4))

  Downloading sentencepiece-0.1.97-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)

Collecting sanic-routing<22.6.0,>=22.3.0 (from sanic==22.6.2->-r requirements.txt (line 1))

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 123.2 MB/s eta 0:00:00
  Downloading sanic_routing-22.3.0-py3-none-any.whl (25 kB)

Collecting httptools>=0.0.10 (from sanic==22.6.2->-r requirements.txt (line 1))

  Downloading httptools-0.6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 441.7/441.7 kB 81.0 MB/s eta 0:00:00Collecting aiofiles>=0.6.0 (from sanic==22.6.2->-r requirements.txt (line 1))

  Downloading aiofiles-23.2.1-py3-none-any.whl (15 kB)

Collecting websockets>=10.0 (from sanic==22.6.2->-r requirements.txt (line 1))

  Downloading websockets-11.0.3-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 130.2/130.2 kB 43.8 MB/s eta 0:00:00Collecting multidict<7.0,>=5.0 (from sanic==22.6.2->-r requirements.txt (line 1))

  Downloading multidict-6.0.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (121 kB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.3/121.3 kB 35.1 MB/s eta 0:00:00Collecting uvloop>=0.5.3 (from sanic==22.6.2->-r requirements.txt (line 1))

  Downloading uvloop-0.17.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB)

Collecting ujson>=1.35 (from sanic==22.6.2->-r requirements.txt (line 1))

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 137.3 MB/s eta 0:00:00
  Downloading ujson-5.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (53 kB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.0/54.0 kB 11.2 MB/s eta 0:00:00Requirement already satisfied: filelock in /opt/conda/lib/python3.8/site-packages (from transformers==4.24.0->-r requirements.txt (line 2)) (3.4.2)

Collecting huggingface-hub<1.0,>=0.10.0 (from transformers==4.24.0->-r requirements.txt (line 2))

  Downloading huggingface_hub-0.16.4-py3-none-any.whl (268 kB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 268.8/268.8 kB 74.6 MB/s eta 0:00:00Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.8/site-packages (from transformers==4.24.0->-r requirements.txt (line 2)) (1.21.2)

Collecting packaging>=20.0 (from transformers==4.24.0->-r requirements.txt (line 2))

  Downloading packaging-23.1-py3-none-any.whl (48 kB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 12.5 MB/s eta 0:00:00Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.8/site-packages (from transformers==4.24.0->-r requirements.txt (line 2)) (6.0)

Collecting regex!=2019.12.17 (from transformers==4.24.0->-r requirements.txt (line 2))

  Downloading regex-2023.8.8-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 774.3/774.3 kB 102.6 MB/s eta 0:00:00Requirement already satisfied: requests in /opt/conda/lib/python3.8/site-packages (from transformers==4.24.0->-r requirements.txt (line 2)) (2.27.1)

Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers==4.24.0->-r requirements.txt (line 2))

  Downloading tokenizers-0.13.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 111.6 MB/s eta 0:00:00Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.8/site-packages (from transformers==4.24.0->-r requirements.txt (line 2)) (4.62.3)

Requirement already satisfied: psutil in /opt/conda/lib/python3.8/site-packages (from accelerate==0.14.0->-r requirements.txt (line 3)) (5.8.0)

Requirement already satisfied: torch>=1.4.0 in /opt/conda/lib/python3.8/site-packages (from accelerate==0.14.0->-r requirements.txt (line 3)) (1.11.0)

Collecting fsspec (from huggingface-hub<1.0,>=0.10.0->transformers==4.24.0->-r requirements.txt (line 2))

  Downloading fsspec-2023.6.0-py3-none-any.whl (163 kB)

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 163.8/163.8 kB 47.8 MB/s eta 0:00:00Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.8/site-packages (from huggingface-hub<1.0,>=0.10.0->transformers==4.24.0->-r requirements.txt (line 2)) (3.10.0.2)

Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.8/site-packages (from requests->transformers==4.24.0->-r requirements.txt (line 2)) (1.26.7)

Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.8/site-packages (from requests->transformers==4.24.0->-r requirements.txt (line 2)) (2021.10.8)

Requirement already satisfied: charset-normalizer~=2.0.0 in /opt/conda/lib/python3.8/site-packages (from requests->transformers==4.24.0->-r requirements.txt (line 2)) (2.0.4)

Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.8/site-packages (from requests->transformers==4.24.0->-r requirements.txt (line 2)) (3.3)

Installing collected packages: tokenizers, sentencepiece, sanic-routing, websockets, uvloop, ujson, regex, packaging, multidict, httptools, fsspec, aiofiles, sanic, huggingface-hub, accelerate, transformers

Successfully installed accelerate-0.14.0 aiofiles-23.2.1 fsspec-2023.6.0 httptools-0.6.0 huggingface-hub-0.16.4 multidict-6.0.4 packaging-23.1 regex-2023.8.8 sanic-22.6.2 sanic-routing-22.3.0 sentencepiece-0.1.97 tokenizers-0.13.3 transformers-4.24.0 ujson-5.8.0 uvloop-0.17.0 websockets-11.0.3

 ---> 60566db86c39


[notice] A new release of pip is available: 23.1.2 -> 23.2.1
[notice] To update, run: pip install --upgrade pip
Step 7/12 : ADD server.py .

 ---> e51342f7cb98

Step 8/12 : ADD download.py .

 ---> 78a5003eb3fb

Step 9/12 : RUN python3 download.py

 ---> Running in 4485b6527ea8

 ---> d41e15294021


Step 10/12 : ADD app.py .

 ---> cd8124cbb93f

Step 11/12 : EXPOSE 8000

 ---> Running in 83578ecec4de

 ---> 9333486faabf

Step 12/12 : CMD python3 -u server.py

 ---> Running in 2f062c4fa47b

 ---> 3bf0205a9b26

Successfully built 3bf0205a9b26

Successfully tagged bananaml/gandalf-repo_t5-xxl_691edf21-d9df-4524-ae45-251b66c45c92:0.0.1

Step 1/5 : ARG imagename

Step 2/5 : FROM ${imagename}

 ---> 3bf0205a9b26

Step 3/5 : ADD 🍌.py 🍌.py

 ---> a7f3a111eed4

Step 4/5 : RUN python3 🍌.py

 ---> Running in 90043ac229f4

Checking for correct Python version...
Python version detected: 3.8
Interpreter location: /opt/conda/bin/python3
Checking Potassium requirements...
Checking installed version of potassium

Package potassium not found.

ERROR: Issue with checking app requirements.
lucataco commented 1 year ago

This Banana repo was for v1 of Banana.dev. This will not work out of the box with the new Potassium cluster(v2 of Banana). For Potassium templates see the updated banana docs OR my other github repositories that start with the word "potas" (for potassium)

gandalf-repo commented 1 year ago

Do you have any example or template for t5-xxl ? I had a look at llama-7B in your repos but I'm concerned about the underlying configuration for t5-xxl.