search

runpod-workers / worker-vllm

The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
MIT License
220 stars 85 forks source link

Runpod #24

Closed saaldigital-cw closed 8 months ago

saaldigital-cw commented 8 months ago

I tried to load the docker image and use in runpod. But i got the following message.

023-12-19T21:52:21.224549804Z Traceback (most recent call last):

2023-12-19T21:52:21.224594641Z File "/handler.py", line 3, in

2023-12-19T21:52:21.224600084Z import runpod

2023-12-19T21:52:21.224605461Z File "/usr/local/lib/python3.11/dist-packages/runpod/init.py", line 6, in

2023-12-19T21:52:21.224610387Z from . import serverless

2023-12-19T21:52:21.224616241Z File "/usr/local/lib/python3.11/dist-packages/runpod/serverless/init.py", line 16, in

2023-12-19T21:52:21.224627070Z from .modules import rp_fastapi

2023-12-19T21:52:21.224631744Z File "/usr/local/lib/python3.11/dist-packages/runpod/serverless/modules/rp_fastapi.py", line 10, in

2023-12-19T21:52:21.224636270Z from fastapi import FastAPI, APIRouter

2023-12-19T21:52:21.224640611Z File "/usr/local/lib/python3.11/dist-packages/fastapi/init.py", line 7, in

2023-12-19T21:52:21.224644971Z from .applications import FastAPI as FastAPI

2023-12-19T21:52:21.224649164Z File "/usr/local/lib/python3.11/dist-packages/fastapi/applications.py", line 16, in

2023-12-19T21:52:21.224653711Z from fastapi import routing

2023-12-19T21:52:21.224691454Z File "/usr/local/lib/python3.11/dist-packages/fastapi/routing.py", line 22, in

2023-12-19T21:52:21.224716007Z from fastapi import params

2023-12-19T21:52:21.224724461Z File "/usr/local/lib/python3.11/dist-packages/fastapi/params.py", line 5, in

2023-12-19T21:52:21.224816202Z from fastapi.openapi.models import Example

2023-12-19T21:52:21.224821635Z File "/usr/local/lib/python3.11/dist-packages/fastapi/openapi/models.py", line 4, in

2023-12-19T21:52:21.224825700Z from fastapi._compat import (

2023-12-19T21:52:21.224829829Z File "/usr/local/lib/python3.11/dist-packages/fastapi/_compat.py", line 20, in

2023-12-19T21:52:21.224913810Z from fastapi.exceptions import RequestErrorModel

2023-12-19T21:52:21.224923693Z File "/usr/local/lib/python3.11/dist-packages/fastapi/exceptions.py", line 3, in

2023-12-19T21:52:21.224950980Z from pydantic import BaseModel, create_model

2023-12-19T21:52:21.224958616Z File "/usr/local/lib/python3.11/dist-packages/pydantic/init.py", line 372, in getattr

2023-12-19T21:52:21.225128094Z module = import_module(module_name, package=package)

2023-12-19T21:52:21.225144639Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2023-12-19T21:52:21.225149112Z File "/usr/lib/python3.11/importlib/init.py", line 126, in import_module

2023-12-19T21:52:21.225160188Z return _bootstrap._gcd_import(name[level:], package, level)

2023-12-19T21:52:21.225196585Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2023-12-19T21:52:21.225211329Z File "/usr/local/lib/python3.11/dist-packages/pydantic/main.py", line 11, in

2023-12-19T21:52:21.225294129Z import pydantic_core

2023-12-19T21:52:21.225303857Z File "/usr/local/lib/python3.11/dist-packages/pydantic_core/init.py", line 30, in

2023-12-19T21:52:21.225314032Z from .core_schema import CoreConfig, CoreSchema, CoreSchemaType, ErrorType

2023-12-19T21:52:21.225320699Z File "/usr/local/lib/python3.11/dist-packages/pydantic_core/core_schema.py", line 15, in

2023-12-19T21:52:21.225353524Z from typing_extensions import deprecated

2023-12-19T21:52:21.225360153Z ImportError: cannot import name 'deprecated' from 'typing_extensions' (/usr/local/lib/python3.11/dist-packages/typing_extensions.py)

Sirri69 commented 8 months ago

Yep, known issue because of locking dependencies

runpod==1.4.2

Absolutely silly behaviour. Pliz fix. ✨

martinkallstrom commented 8 months ago

Finding same

alpayariyak commented 8 months ago

Should be fixed now

saaldigital-cw commented 8 months ago

Will there be a new docker image or where it is fixed?