error: with prompt: 介绍下大熊猫

[wonderisland]

Generating with prompt: <|system|> User will provide you with a text instruction. Do it step by step. First, think about the instruction and respond in a interleaved manner, with 13 text token followed by 26 audio tokens.<|user|> 介绍下大熊猫<|assistant|>streaming_transcription

thread '' panicked at 'index out of bounds: the len is 187 but the index is 187', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/regex-1.8.1/src/dfa.rs:1415:45 stack backtrace: 0: rust_begin_unwind at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/std/src/panicking.rs:579:5 1: core::panicking::panic_fmt at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/panicking.rs:64:14 2: core::panicking::panic_bounds_check at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/panicking.rs:159:5 3: regex::re_unicode::Regex::find_at 4: fancy_regex::Regex::find_from_pos_with_option_flags 5: fancy_regex::Regex::find_from_pos 6: _tiktoken::CoreBPE::_encode_native 7: pyo3::marker::Python::allow_threads 8: _tiktoken::CoreBPE::encode 9: std::panicking::try 10: tiktoken::::::pymethod_encode 11: 12: _PyEval_EvalFrameDefault 13: _PyFunction_Vectorcall 14: _PyEval_EvalFrameDefault 15: _PyFunction_Vectorcall 16: _PyEval_EvalFrameDefault 17: 18: _PyEval_EvalFrameDefault 19: _PyFunction_Vectorcall 20: _PyEval_EvalFrameDefault 21: 22: PyObject_Call 23: _PyEval_EvalFrameDefault 24: 25: PyObject_Call 26: _PyEval_EvalFrameDefault 27: 28: PyObject_Call 29: _PyEval_EvalFrameDefault 30: _PyFunction_Vectorcall 31: _PyObject_FastCallDictTstate 32: _PyObject_Call_Prepend 33: 34: _PyObject_MakeTpCall 35: _PyEval_EvalFrameDefault 36: 37: 38: 39: _PyEval_EvalFrameDefault 40: 41: _PyEval_EvalFrameDefault 42: 43: 44: _PyEval_EvalFrameDefault 45: _PyFunction_Vectorcall 46: 47: 48: 49: _PyEval_EvalFrameDefault 50: _PyFunction_Vectorcall 51: _PyEval_EvalFrameDefault 52: _PyFunction_Vectorcall 53: _PyEval_EvalFrameDefault 54: 55: 56: 57: 58: clone note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace. ERROR: Exception in ASGI application Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 259, in call__ await wrap(partial(self.listen_for_disconnect, receive)) File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 255, in wrap await func() File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 232, in listen_for_disconnect message = await receive() File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/h11_impl.py", line 534, in receive await self.message_event.wait() File "/usr/lib/python3.10/asyncio/locks.py", line 214, in wait await fut asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f5b146d2320

During handling of the above exception, another exception occurred:

• Exception Group Traceback (most recent call last): | File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi | result = await app( # type: ignore[func-returns-value] | File "/usr/local/lib/python3.10/dist-packages/uvicorn/middleware/proxy_headers.py", line 60, in call | return await self.app(scope, receive, send) | File "/usr/local/lib/python3.10/dist-packages/fastapi/applications.py", line 1054, in call | await super().call(scope, receive, send) | File "/usr/local/lib/python3.10/dist-packages/starlette/applications.py", line 113, in call | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 165, in call | await self.app(scope, receive, _send) | File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 62, in call | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) | File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 42, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 715, in call | await self.middleware_stack(scope, receive, send) | File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 735, in app | await route.handle(scope, receive, send) | File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 288, in handle | await self.app(scope, receive, send) | File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 76, in app | await wrap_app_handling_exceptions(app, request)(scope, receive, send) | File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 42, in wrapped_app | await app(scope, receive, sender) | File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 74, in app | await response(scope, receive, send) | File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 252, in call | async with anyio.create_task_group() as task_group: | File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 763, in aexit | raise BaseExceptionGroup( | exceptiongroup.BaseExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 255, in wrap | await func() | File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 244, in stream_response | async for chunk in self.body_iterator: | File "/usr/local/lib/python3.10/dist-packages/starlette/concurrency.py", line 62, in iterate_in_threadpool | yield await anyio.to_thread.run_sync(_next, as_iterator) | File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 56, in run_sync | return await get_async_backend().run_sync_in_worker_thread( | File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 2441, in run_sync_in_worker_thread | return await future | File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 943, in run | result = context.run(func, *args) | File "/usr/local/lib/python3.10/dist-packages/starlette/concurrency.py", line 51, in _next | return next(iterator) | File "/mnt/data/hectorgao/workspace/GLM-4-Voice/model_server.py", line 87, in generate_stream_gate | for x in self.generate_stream(params): | File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 35, in generator_context | response = gen.send(None) | File "/mnt/data/hectorgao/workspace/GLM-4-Voice/model_server.py", line 74, in generate_stream | inputs = tokenizer([prompt], return_tensors="pt") | File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py", line 3055, in call | encodings = self._call_one(text=text, text_pair=text_pair, all_kwargs) | File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py", line 3142, in _call_one | return self.batch_encode_plus( | File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py", line 3338, in batch_encode_plus | return self._batch_encode_plus( | File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils.py", line 882, in _batch_encode_plus | first_ids = get_input_ids(ids) | File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils.py", line 849, in get_input_ids | tokens = self.tokenize(text, kwargs) | File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils.py", line 695, in tokenize | tokenized_text.extend(self._tokenize(token)) | File "/root/.cache/huggingface/modules/transformers_modules/tokenization_chatglm.py", line 82, in _tokenize | ids = self.tokenizer.encode(text) | File "/usr/local/lib/python3.10/dist-packages/tiktoken/core.py", line 120, in encode | return self._core_bpe.encode(text, allowed_special) | pyo3_runtime.PanicException: index out of bounds: the len is 187 but the index is 187

[wonderisland]

tiktoken package version is too old, update it