[Bug]: GPUExecutor throwing 'TypeError: 'type' object is not subscriptable' on 0.5.3

Your current environment

Package                   Version
------------------------- --------------------
aiohttp                   3.9.5
aiosignal                 1.3.1
annotated-types           0.6.0
anyio                     4.3.0
async-timeout             4.0.3
attrs                     23.2.0
bitsandbytes              0.42.0
certifi                   2019.11.28
chardet                   3.0.4
charset-normalizer        3.3.2
click                     8.1.7
cloudpickle               3.0.0
cmake                     3.29.3
colorlog                  6.8.2
dbus-python               1.2.16
diskcache                 5.6.3
distro                    1.9.0
dnspython                 2.6.1
einops                    0.8.0
email-validator           2.1.1
exceptiongroup            1.2.1
fastapi                   0.111.0
fastapi-cli               0.0.3
filelock                  3.14.0
frozenlist                1.4.1
fschat                    0.2.36
fsspec                    2024.3.1
h11                       0.14.0
hf-transfer               0.1.6
httpcore                  1.0.5
httpx                     0.27.0
huggingface-hub           0.23.0
idna                      2.8
importlib-metadata        7.1.0
importlib-resources       6.4.0
interegular               0.3.3
jinja2                    3.1.4
joblib                    1.4.2
jsonschema                4.22.0
jsonschema-specifications 2023.12.1
lark                      1.1.8
llvmlite                  0.41.1
lm-format-enforcer        0.10.1
loguru                    0.7.2
markdown-it-py            3.0.0
markdown2                 2.4.13
MarkupSafe                2.1.5
mdurl                     0.1.2
mpmath                    1.3.0
msgpack                   1.0.8
multidict                 6.0.5
nest-asyncio              1.6.0
networkx                  3.1
nh3                       0.2.17
ninja                     1.11.1.1
numba                     0.58.1
numpy                     1.24.4
nvidia-cublas-cu12        12.1.3.1
nvidia-cuda-cupti-cu12    12.1.105
nvidia-cuda-nvrtc-cu12    12.1.105
nvidia-cuda-runtime-cu12  12.1.105
nvidia-cudnn-cu12         8.9.2.26
nvidia-cufft-cu12         11.0.2.54
nvidia-curand-cu12        10.3.2.106
nvidia-cusolver-cu12      11.4.5.107
nvidia-cusparse-cu12      12.1.0.106
nvidia-nccl-cu12          2.20.5
nvidia-nvjitlink-cu12     12.4.127
nvidia-nvtx-cu12          12.1.105
openai                    1.29.0
orjson                    3.10.3
outlines                  0.0.41
packaging                 24.0
pillow                    10.3.0
pip                       20.0.2
pkgutil-resolve-name      1.3.10
prometheus-client         0.20.0
prompt-toolkit            3.0.43
protobuf                  5.26.1
psutil                    5.9.8
pydantic                  2.7.1
pydantic-core             2.18.2
pygments                  2.18.0
PyGObject                 3.36.0
pynvml                    11.5.0
python-apt                2.0.1+ubuntu0.20.4.1
python-multipart          0.0.9
PyYAML                    6.0.1
ray                       2.10.0
referencing               0.35.1
regex                     2024.5.10
requests                  2.31.0
requests-unixsocket       0.2.0
rich                      13.7.1
rpds-py                   0.18.1
safetensors               0.4.3
scikit-learn              1.3.2
scipy                     1.10.1
sentence-transformers     2.7.0
sentencepiece             0.2.0
setuptools                45.2.0
shellingham               1.5.4
shortuuid                 1.0.13
six                       1.14.0
sniffio                   1.3.1
starlette                 0.37.2
svgwrite                  1.4.3
sympy                     1.12
threadpoolctl             3.5.0
tiktoken                  0.6.0
tokenizers                0.19.1
torch                     2.3.0
tqdm                      4.66.4
transformers              4.40.0
triton                    2.3.0
typer                     0.12.3
typing-extensions         4.11.0
ujson                     5.10.0
urllib3                   1.25.8
uvicorn                   0.29.0
wavedrom                  2.0.3.post3
wcwidth                   0.2.13
wheel                     0.34.2
xformers                  0.0.26.post1
yarl                      1.9.4
zipp                      3.18.1

🐛 Describe the bug

i ran below code to create a simple api server:

python3 -m aphrodite.endpoints.openai.api_server \
--model microsoft/Phi-3-mini-4k-instruct \
--trust-remote-code

and got below error

/usr/local/lib/python3.8/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/root/aphrodite-engine/aphrodite/endpoints/openai/api_server.py", line 562, in <module>
    run_server(args)
  File "/root/aphrodite-engine/aphrodite/endpoints/openai/api_server.py", line 519, in run_server
    engine = AsyncAphrodite.from_engine_args(engine_args)
  File "/root/aphrodite-engine/aphrodite/engine/async_aphrodite.py", line 355, in from_engine_args
    from aphrodite.executor.gpu_executor import GPUExecutorAsync
  File "/root/aphrodite-engine/aphrodite/executor/gpu_executor.py", line 16, in <module>
    class GPUExecutor(ExecutorBase):
  File "/root/aphrodite-engine/aphrodite/executor/gpu_executor.py", line 92, in GPUExecutor
    def determine_num_available_blocks(self) -> tuple[int, int]:
TypeError: 'type' object is not subscriptable

i tried install via both below but both got the error $ pip install -U aphrodite-engine --extra-index-url https://downloads.pygmalion.chat/whl or build from source

Not sure if that specific to my env?

PygmalionAI / aphrodite-engine

[Bug]: GPUExecutor throwing 'TypeError: 'type' object is not subscriptable' on 0.5.3 #470

Your current environment

🐛 Describe the bug