Tutorial not running - Githubissues

Hello, when trying to make bonito work for my application, I ran into some issues. To try and find the problems I wanted to run the google colab example and slowly make changes to see where my mistake was. However the tutorial colab doesn't seem to work in its original form. I have been considering trying an older version of bonito but I am not sure how to do this when cloning from git. Any help would be greatly appreciated. This is the output from the installation where a pip Error occurs:

Cloning into 'bonito'...
remote: Enumerating objects: 91, done.
remote: Counting objects: 100% (46/46), done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 91 (delta 36), reused 26 (delta 26), pack-reused 45
Receiving objects: 100% (91/91), 784.34 KiB | 34.10 MiB/s, done.
Resolving deltas: 100% (38/38), done.
Processing ./bonito
  Preparing metadata (setup.py) ... done
Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (from bonito==0.0.2) (4.41.0)
Collecting datasets (from bonito==0.0.2)
  Downloading datasets-2.19.1-py3-none-any.whl (542 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 542.0/542.0 kB 7.9 MB/s eta 0:00:00
Collecting vllm (from bonito==0.0.2)
  Downloading vllm-0.4.2-cp310-cp310-manylinux1_x86_64.whl (67.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.7/67.7 MB 11.6 MB/s eta 0:00:00
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from datasets->bonito==0.0.2) (3.14.0)
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets->bonito==0.0.2) (1.25.2)
Requirement already satisfied: pyarrow>=12.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets->bonito==0.0.2) (14.0.2)
Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets->bonito==0.0.2) (0.6)
Collecting dill<0.3.9,>=0.3.0 (from datasets->bonito==0.0.2)
  Downloading dill-0.3.8-py3-none-any.whl (116 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.3/116.3 kB 17.9 MB/s eta 0:00:00
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets->bonito==0.0.2) (2.0.3)
Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from datasets->bonito==0.0.2) (2.31.0)
Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from datasets->bonito==0.0.2) (4.66.4)
Collecting xxhash (from datasets->bonito==0.0.2)
  Downloading xxhash-3.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (194 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 194.1/194.1 kB 25.0 MB/s eta 0:00:00
Collecting multiprocess (from datasets->bonito==0.0.2)
  Downloading multiprocess-0.70.16-py310-none-any.whl (134 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.8/134.8 kB 14.7 MB/s eta 0:00:00
Requirement already satisfied: fsspec[http]<=2024.3.1,>=2023.1.0 in /usr/local/lib/python3.10/dist-packages (from datasets->bonito==0.0.2) (2023.6.0)
Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets->bonito==0.0.2) (3.9.5)
Requirement already satisfied: huggingface-hub>=0.21.2 in /usr/local/lib/python3.10/dist-packages (from datasets->bonito==0.0.2) (0.23.1)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets->bonito==0.0.2) (24.0)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets->bonito==0.0.2) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers->bonito==0.0.2) (2023.12.25)
Requirement already satisfied: tokenizers<0.20,>=0.19 in /usr/local/lib/python3.10/dist-packages (from transformers->bonito==0.0.2) (0.19.1)
Requirement already satisfied: safetensors>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from transformers->bonito==0.0.2) (0.4.3)
Requirement already satisfied: cmake>=3.21 in /usr/local/lib/python3.10/dist-packages (from vllm->bonito==0.0.2) (3.27.9)
Collecting ninja (from vllm->bonito==0.0.2)
  Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (307 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 307.2/307.2 kB 18.8 MB/s eta 0:00:00
Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from vllm->bonito==0.0.2) (5.9.5)
Requirement already satisfied: sentencepiece in /usr/local/lib/python3.10/dist-packages (from vllm->bonito==0.0.2) (0.1.99)
Requirement already satisfied: py-cpuinfo in /usr/local/lib/python3.10/dist-packages (from vllm->bonito==0.0.2) (9.0.0)
Collecting fastapi (from vllm->bonito==0.0.2)
  Downloading fastapi-0.111.0-py3-none-any.whl (91 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 92.0/92.0 kB 10.3 MB/s eta 0:00:00
Collecting openai (from vllm->bonito==0.0.2)
  Downloading openai-1.30.3-py3-none-any.whl (320 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 320.6/320.6 kB 23.8 MB/s eta 0:00:00
Collecting uvicorn[standard] (from vllm->bonito==0.0.2)
  Downloading uvicorn-0.29.0-py3-none-any.whl (60 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.8/60.8 kB 4.6 MB/s eta 0:00:00
Requirement already satisfied: pydantic>=2.0 in /usr/local/lib/python3.10/dist-packages (from vllm->bonito==0.0.2) (2.7.1)
Requirement already satisfied: prometheus-client>=0.18.0 in /usr/local/lib/python3.10/dist-packages (from vllm->bonito==0.0.2) (0.20.0)
Collecting prometheus-fastapi-instrumentator>=7.0.0 (from vllm->bonito==0.0.2)
  Downloading prometheus_fastapi_instrumentator-7.0.0-py3-none-any.whl (19 kB)
Collecting tiktoken==0.6.0 (from vllm->bonito==0.0.2)
  Downloading tiktoken-0.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 60.5 MB/s eta 0:00:00
Collecting lm-format-enforcer==0.9.8 (from vllm->bonito==0.0.2)
  Downloading lm_format_enforcer-0.9.8-py3-none-any.whl (40 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.9/40.9 kB 5.6 MB/s eta 0:00:00
Collecting outlines==0.0.34 (from vllm->bonito==0.0.2)
  Downloading outlines-0.0.34-py3-none-any.whl (76 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 76.5/76.5 kB 3.5 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from vllm->bonito==0.0.2) (4.11.0)
Collecting ray>=2.9 (from vllm->bonito==0.0.2)
  Downloading ray-2.23.0-cp310-cp310-manylinux2014_x86_64.whl (65.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.7/65.7 MB 7.4 MB/s eta 0:00:00
Collecting nvidia-ml-py (from vllm->bonito==0.0.2)
  Downloading nvidia_ml_py-12.550.52-py3-none-any.whl (39 kB)
Collecting vllm-nccl-cu12<2.19,>=2.18 (from vllm->bonito==0.0.2)
  Downloading vllm_nccl_cu12-2.18.1.0.4.0.tar.gz (6.2 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: torch==2.3.0 in /usr/local/lib/python3.10/dist-packages (from vllm->bonito==0.0.2) (2.3.0+cu121)
Collecting xformers==0.0.26.post1 (from vllm->bonito==0.0.2)
  Downloading xformers-0.0.26.post1-cp310-cp310-manylinux2014_x86_64.whl (222.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 222.7/222.7 MB 6.0 MB/s eta 0:00:00
Collecting interegular>=0.3.2 (from lm-format-enforcer==0.9.8->vllm->bonito==0.0.2)
  Downloading interegular-0.3.3-py37-none-any.whl (23 kB)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from outlines==0.0.34->vllm->bonito==0.0.2) (3.1.4)
Collecting lark (from outlines==0.0.34->vllm->bonito==0.0.2)
  Downloading lark-1.1.9-py3-none-any.whl (111 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 111.7/111.7 kB 18.4 MB/s eta 0:00:00
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.10/dist-packages (from outlines==0.0.34->vllm->bonito==0.0.2) (1.6.0)
Requirement already satisfied: cloudpickle in /usr/local/lib/python3.10/dist-packages (from outlines==0.0.34->vllm->bonito==0.0.2) (2.2.1)
Collecting diskcache (from outlines==0.0.34->vllm->bonito==0.0.2)
  Downloading diskcache-5.6.3-py3-none-any.whl (45 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.5/45.5 kB 7.5 MB/s eta 0:00:00
Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from outlines==0.0.34->vllm->bonito==0.0.2) (1.11.4)
Requirement already satisfied: numba in /usr/local/lib/python3.10/dist-packages (from outlines==0.0.34->vllm->bonito==0.0.2) (0.58.1)
Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from outlines==0.0.34->vllm->bonito==0.0.2) (1.4.2)
Requirement already satisfied: referencing in /usr/local/lib/python3.10/dist-packages (from outlines==0.0.34->vllm->bonito==0.0.2) (0.35.1)
Requirement already satisfied: jsonschema in /usr/local/lib/python3.10/dist-packages (from outlines==0.0.34->vllm->bonito==0.0.2) (4.19.2)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch==2.3.0->vllm->bonito==0.0.2) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch==2.3.0->vllm->bonito==0.0.2) (3.3)
Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch==2.3.0->vllm->bonito==0.0.2)
  Using cached nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)
Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch==2.3.0->vllm->bonito==0.0.2)
  Using cached nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)
Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch==2.3.0->vllm->bonito==0.0.2)
  Using cached nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)
Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch==2.3.0->vllm->bonito==0.0.2)
  Using cached nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)
Collecting nvidia-cublas-cu12==12.1.3.1 (from torch==2.3.0->vllm->bonito==0.0.2)
  Using cached nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)
Collecting nvidia-cufft-cu12==11.0.2.54 (from torch==2.3.0->vllm->bonito==0.0.2)
  Using cached nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)
Collecting nvidia-curand-cu12==10.3.2.106 (from torch==2.3.0->vllm->bonito==0.0.2)
  Using cached nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)
Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch==2.3.0->vllm->bonito==0.0.2)
  Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)
Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch==2.3.0->vllm->bonito==0.0.2)
  Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)
Collecting nvidia-nccl-cu12==2.20.5 (from torch==2.3.0->vllm->bonito==0.0.2)
  Using cached nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_x86_64.whl (176.2 MB)
Collecting nvidia-nvtx-cu12==12.1.105 (from torch==2.3.0->vllm->bonito==0.0.2)
  Using cached nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)
Requirement already satisfied: triton==2.3.0 in /usr/local/lib/python3.10/dist-packages (from torch==2.3.0->vllm->bonito==0.0.2) (2.3.0)
Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch==2.3.0->vllm->bonito==0.0.2)
  Downloading nvidia_nvjitlink_cu12-12.5.40-py3-none-manylinux2014_x86_64.whl (21.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.3/21.3 MB 71.3 MB/s eta 0:00:00
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->bonito==0.0.2) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->bonito==0.0.2) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->bonito==0.0.2) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->bonito==0.0.2) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->bonito==0.0.2) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->bonito==0.0.2) (4.0.3)
Collecting starlette<1.0.0,>=0.30.0 (from prometheus-fastapi-instrumentator>=7.0.0->vllm->bonito==0.0.2)
  Downloading starlette-0.37.2-py3-none-any.whl (71 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.9/71.9 kB 10.9 MB/s eta 0:00:00
Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic>=2.0->vllm->bonito==0.0.2) (0.7.0)
Requirement already satisfied: pydantic-core==2.18.2 in /usr/local/lib/python3.10/dist-packages (from pydantic>=2.0->vllm->bonito==0.0.2) (2.18.2)
Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from ray>=2.9->vllm->bonito==0.0.2) (8.1.7)
Requirement already satisfied: msgpack<2.0.0,>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from ray>=2.9->vllm->bonito==0.0.2) (1.0.8)
Requirement already satisfied: protobuf!=3.19.5,>=3.15.3 in /usr/local/lib/python3.10/dist-packages (from ray>=2.9->vllm->bonito==0.0.2) (3.20.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets->bonito==0.0.2) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets->bonito==0.0.2) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets->bonito==0.0.2) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets->bonito==0.0.2) (2024.2.2)
Collecting fastapi-cli>=0.0.2 (from fastapi->vllm->bonito==0.0.2)
  Downloading fastapi_cli-0.0.4-py3-none-any.whl (9.5 kB)
Collecting httpx>=0.23.0 (from fastapi->vllm->bonito==0.0.2)
  Downloading httpx-0.27.0-py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.6/75.6 kB 12.7 MB/s eta 0:00:00
Collecting python-multipart>=0.0.7 (from fastapi->vllm->bonito==0.0.2)
  Downloading python_multipart-0.0.9-py3-none-any.whl (22 kB)
Collecting ujson!=4.0.2,!=4.1.0,!=4.2.0,!=4.3.0,!=5.0.0,!=5.1.0,>=4.0.1 (from fastapi->vllm->bonito==0.0.2)
  Downloading ujson-5.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (53 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.6/53.6 kB 8.7 MB/s eta 0:00:00
Collecting orjson>=3.2.1 (from fastapi->vllm->bonito==0.0.2)
  Downloading orjson-3.10.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 142.5/142.5 kB 21.9 MB/s eta 0:00:00
Collecting email_validator>=2.0.0 (from fastapi->vllm->bonito==0.0.2)
  Downloading email_validator-2.1.1-py3-none-any.whl (30 kB)
Collecting h11>=0.8 (from uvicorn[standard]->vllm->bonito==0.0.2)
  Downloading h11-0.14.0-py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 9.7 MB/s eta 0:00:00
Collecting httptools>=0.5.0 (from uvicorn[standard]->vllm->bonito==0.0.2)
  Downloading httptools-0.6.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (341 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 341.4/341.4 kB 41.8 MB/s eta 0:00:00
Collecting python-dotenv>=0.13 (from uvicorn[standard]->vllm->bonito==0.0.2)
  Downloading python_dotenv-1.0.1-py3-none-any.whl (19 kB)
Collecting uvloop!=0.15.0,!=0.15.1,>=0.14.0 (from uvicorn[standard]->vllm->bonito==0.0.2)
  Downloading uvloop-0.19.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 81.5 MB/s eta 0:00:00
Collecting watchfiles>=0.13 (from uvicorn[standard]->vllm->bonito==0.0.2)
  Downloading watchfiles-0.21.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 86.3 MB/s eta 0:00:00
Collecting websockets>=10.4 (from uvicorn[standard]->vllm->bonito==0.0.2)
  Downloading websockets-12.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 130.2/130.2 kB 17.4 MB/s eta 0:00:00
Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai->vllm->bonito==0.0.2) (3.7.1)
Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai->vllm->bonito==0.0.2) (1.7.0)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai->vllm->bonito==0.0.2) (1.3.1)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets->bonito==0.0.2) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets->bonito==0.0.2) (2023.4)
Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets->bonito==0.0.2) (2024.1)
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai->vllm->bonito==0.0.2) (1.2.1)
Collecting dnspython>=2.0.0 (from email_validator>=2.0.0->fastapi->vllm->bonito==0.0.2)
  Downloading dnspython-2.6.1-py3-none-any.whl (307 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 307.7/307.7 kB 42.7 MB/s eta 0:00:00
Collecting typer>=0.12.3 (from fastapi-cli>=0.0.2->fastapi->vllm->bonito==0.0.2)
  Downloading typer-0.12.3-py3-none-any.whl (47 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 47.2/47.2 kB 7.4 MB/s eta 0:00:00
Collecting httpcore==1.* (from httpx>=0.23.0->fastapi->vllm->bonito==0.0.2)
  Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 12.5 MB/s eta 0:00:00
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->outlines==0.0.34->vllm->bonito==0.0.2) (2.1.5)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->datasets->bonito==0.0.2) (1.16.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema->outlines==0.0.34->vllm->bonito==0.0.2) (2023.12.1)
Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema->outlines==0.0.34->vllm->bonito==0.0.2) (0.18.1)
Requirement already satisfied: llvmlite<0.42,>=0.41.0dev0 in /usr/local/lib/python3.10/dist-packages (from numba->outlines==0.0.34->vllm->bonito==0.0.2) (0.41.1)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch==2.3.0->vllm->bonito==0.0.2) (1.3.0)
Collecting shellingham>=1.3.0 (from typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm->bonito==0.0.2)
  Downloading shellingham-1.5.4-py2.py3-none-any.whl (9.8 kB)
Requirement already satisfied: rich>=10.11.0 in /usr/local/lib/python3.10/dist-packages (from typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm->bonito==0.0.2) (13.7.1)
Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich>=10.11.0->typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm->bonito==0.0.2) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich>=10.11.0->typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm->bonito==0.0.2) (2.16.1)
Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich>=10.11.0->typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm->bonito==0.0.2) (0.1.2)
Building wheels for collected packages: bonito, vllm-nccl-cu12
  Building wheel for bonito (setup.py) ... done
  Created wheel for bonito: filename=bonito-0.0.2-py3-none-any.whl size=4585 sha256=ad2c9ec579a44fbef608e6babff389ec446e2e1946495a5197a0cef1900431c6
  Stored in directory: /tmp/pip-ephem-wheel-cache-a1qhngbb/wheels/c5/fe/de/e0c4849775dee927ba7352098bc3e060482a9ac937dde7f9a3
  Building wheel for vllm-nccl-cu12 (setup.py) ... done
  Created wheel for vllm-nccl-cu12: filename=vllm_nccl_cu12-2.18.1.0.4.0-py3-none-any.whl size=5418 sha256=a60cfd55ede786f0554b93235479600d23a44f1dd8b8b5fcc70ad018e23c692c
  Stored in directory: /root/.cache/pip/wheels/d1/28/b5/e99e6ea84b08c0bf19a218d408316e55e02ff725d3616fb79d
Successfully built bonito vllm-nccl-cu12
Installing collected packages: vllm-nccl-cu12, nvidia-ml-py, ninja, xxhash, websockets, uvloop, ujson, shellingham, python-multipart, python-dotenv, orjson, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, lark, interegular, httptools, h11, dnspython, diskcache, dill, watchfiles, uvicorn, tiktoken, starlette, nvidia-cusparse-cu12, nvidia-cudnn-cu12, multiprocess, httpcore, email_validator, typer, prometheus-fastapi-instrumentator, nvidia-cusolver-cu12, lm-format-enforcer, httpx, ray, openai, fastapi-cli, datasets, xformers, outlines, fastapi, vllm, bonito
  Attempting uninstall: typer
    Found existing installation: typer 0.9.4
    Uninstalling typer-0.9.4:
      Successfully uninstalled typer-0.9.4
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
spacy 3.7.4 requires typer<0.10.0,>=0.3.0, but you have typer 0.12.3 which is incompatible.
weasel 0.3.4 requires typer<0.10.0,>=0.3.0, but you have typer 0.12.3 which is incompatible.
Successfully installed bonito-0.0.2 datasets-2.19.1 dill-0.3.8 diskcache-5.6.3 dnspython-2.6.1 email_validator-2.1.1 fastapi-0.111.0 fastapi-cli-0.0.4 h11-0.14.0 httpcore-1.0.5 httptools-0.6.1 httpx-0.27.0 interegular-0.3.3 lark-1.1.9 lm-format-enforcer-0.9.8 multiprocess-0.70.16 ninja-1.11.1.1 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-ml-py-12.550.52 nvidia-nccl-cu12-2.20.5 nvidia-nvjitlink-cu12-12.5.40 nvidia-nvtx-cu12-12.1.105 openai-1.30.3 orjson-3.10.3 outlines-0.0.34 prometheus-fastapi-instrumentator-7.0.0 python-dotenv-1.0.1 python-multipart-0.0.9 ray-2.23.0 shellingham-1.5.4 starlette-0.37.2 tiktoken-0.6.0 typer-0.12.3 ujson-5.10.0 uvicorn-0.29.0 uvloop-0.19.0 vllm-0.4.2 vllm-nccl-cu12-2.18.1.0.4.0 watchfiles-0.21.0 websockets-12.0 xformers-0.0.26.post1 xxhash-3.4.1

The code keeps running without further problems until the "Generate the synthetic instructions"-section. Here again the output:

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.

---------------------------------------------------------------------------

AssertionError                            Traceback (most recent call last)

[<ipython-input-6-0ad9bfde7270>](https://localhost:8080/#) in <cell line: 7>()
      5 # Generate synthetic instruction tuning dataset
      6 sampling_params = {'max_new_tokens':256, 'top_p':0.95, 'temperature':0.7, 'num_return_sequences':1}
----> 7 synthetic_dataset = bonito.generate_task(
      8     unannotated_paragraph,
      9     task_type="nli",

18 frames

[/usr/local/lib/python3.10/dist-packages/awq/modules/fused/norm.py](https://localhost:8080/#) in forward(self, x)
     17 
     18     def forward(self, x):
---> 19         assert AWQ_INSTALLED, (
     20             "AWQ kernels could not be loaded. "
     21             "Please install them from https://github.com/casper-hansen/AutoAWQ_kernels"

AssertionError: AWQ kernels could not be loaded. Please install them from https://github.com/casper-hansen/AutoAWQ_kernels

It did say it successfully installed the autoawq-kernels-0.0.6+cu122 so I am not sure how I should try to solve this problem.

BatsResearch / bonito

Tutorial not running #24

Additional Details

Additional Details