ghost
commented
1 month ago Open Clint-chan opened 1 month ago
ghost
commented
1 month ago 
Clint-chan
commented
1 month ago LOG_DIR="/raid/xinference/modelscope/hub/qwen/Qwen2-72B-Instruct/logs"; \ mkdir -p "$LOG_DIR"; \ LOG_FILE="$LOGDIR/vllm$(date +%Y%m%d).log"; \ echo "Logging to $LOG_FILE" >&2; \ conda run -n vllm_062 vllm serve \ /raid/Qwen2-72B-Instruct \ --port 20005 \ --dtype bfloat16 \ --tensor-parallel-size 4 \ --max-num-seqs 1024 \ --gpu-memory-utilization 0.85 \ --max-num-batched-tokens 8192 \ --max-model-len 8192 \ --block-size 32 \ --enforce-eager \ --enable_chunked_prefill=True \
"$LOG_FILE" 2>&1' &
Clint-chan
commented
1 month ago 分析一下我的日志：INFO 09-29 08:48:42 logger.py:36] Received request chat-312348aecf9e42ec91de81da41e091db: prompt: '<|im_start|>system\n你是一位商品名词分析专家，请从描述内容中分析是否包含与输入的产品关键词含义相同。请使用Json 的格式返回如下结果，结果的值只允许有包含和不包含。格式 如下：{“result”:"结果"}<|im_end|>\n<|im_start|>user\nFIBER CABLE是否包含在哪下内容中，内容如下：None<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.7, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=None, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: [151644, 8948, 198, 56568, 109182, 45943, 113046, 101042, 101057, 37945, 45181, 53481, 43815, 15946, 101042, 64471, 102298, 57218, 31196, 104310, 105291, 109091, 102486, 1773, 14880, 37029, 5014, 43589, 68805, 31526, 104506, 59151, 3837, 59151, 9370, 25511, 91680, 102496, 18830, 102298, 33108, 16530, 102298, 1773, 68805, 69372, 16872, 5122, 90, 2073, 1382, 854, 2974, 59151, 9207, 151645, 198, 151644, 872, 198, 37, 3256, 640, 356, 3494, 64471, 102298, 109333, 16872, 43815, 15946, 3837, 43815, 104506, 5122, 4064, 151645, 198, 151644, 77091, 198], lora_request: None, prompt_adapter_request: None. INFO: 61.171.72.231:32674 - "POST /v1/chat/completions HTTP/1.1" 200 OK ERROR: Exception in ASGI application Traceback (most recent call last): File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/responses.py", line 265, in call await wrap(partial(self.listen_for_disconnect, receive)) File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/responses.py", line 261, in wrap await func() File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/responses.py", line 238, in listen_for_disconnect message = await receive() File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 553, in receive await self.message_event.wait() File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/asyncio/locks.py", line 214, in wait await fut asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f5c49584a90
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 399, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call
return await self.app(scope, receive, send)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in call
await super().call(scope, receive, send)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/applications.py", line 123, in call
await self.middleware_stack(scope, receive, send)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in call
raise exc
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in call
await self.app(scope, receive, _send)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in call
await self.app(scope, receive, send)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in call
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/routing.py", line 756, in call
await self.middleware_stack(scope, receive, send)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
await route.handle(scope, receive, send)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
await self.app(scope, receive, send)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/routing.py", line 75, in app
await response(scope, receive, send)
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/starlette/responses.py", line 258, in call
async with anyio.create_task_group() as task_group:
File "/raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 680, in aexit
raise BaseExceptionGroup(
exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
(VllmWorkerProcess pid=3223236) WARNING 09-29 08:48:42 shm_broadcast.py:404] No available block found in 60 second.
(VllmWorkerProcess pid=3223237) WARNING 09-29 08:48:42 shm_broadcast.py:404] No available block found in 60 second.
(VllmWorkerProcess pid=3223235) WARNING 09-29 08:48:42 shm_broadcast.py:404] No available block found in 60 second.
[rank2]:[E ProcessGroupNCCL.cpp:563] [Rank 2] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=29100340, OpType=GATHER, NumelIn=38016, NumelOut=0, Timeout(ms)=600000) ran for 600043 milliseconds before timing out.
[rank2]:[E ProcessGroupNCCL.cpp:1537] [PG 2 Rank 2] Timeout at NCCL work: 29100340, last enqueued NCCL work: 29100340, last completed NCCL work: 29100339.
[rank2]:[E ProcessGroupNCCL.cpp:577] [Rank 2] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data.
[rank2]:[E ProcessGroupNCCL.cpp:583] [Rank 2] To avoid data inconsistency, we are taking the entire process down.
[rank2]:[E ProcessGroupNCCL.cpp:1414] [PG 2 Rank 2] Process group watchdog thread terminated with exception: [Rank 2] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=29100340, OpType=GATHER, NumelIn=38016, NumelOut=0, Timeout(ms)=600000) ran for 600043 milliseconds before timing out.
Exception raised from checkTimeout at ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:565 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7f5cf7ba9897 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libc10.so)
frame #1: c10d::ProcessGroupNCCL::WorkNCCL::checkTimeout(std::optional<std::chrono::duration<long, std::ratio<1l, 1000l> > >) + 0x1d2 (0x7f5cf8e82c62 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
frame #2: c10d::ProcessGroupNCCL::watchdogHandler() + 0x1a0 (0x7f5cf8e87a80 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
frame #3: c10d::ProcessGroupNCCL::ncclCommWatchdog() + 0x10c (0x7f5cf8e88dcc in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
frame #4:
[rank3]:[E ProcessGroupNCCL.cpp:563] [Rank 3] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=29100340, OpType=GATHER, NumelIn=38016, NumelOut=0, Timeout(ms)=600000) ran for 600051 milliseconds before timing out.
[rank3]:[E ProcessGroupNCCL.cpp:1537] [PG 2 Rank 3] Timeout at NCCL work: 29100340, last enqueued NCCL work: 29100340, last completed NCCL work: 29100339.
[rank3]:[E ProcessGroupNCCL.cpp:577] [Rank 3] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data.
[rank3]:[E ProcessGroupNCCL.cpp:583] [Rank 3] To avoid data inconsistency, we are taking the entire process down.
[rank3]:[E ProcessGroupNCCL.cpp:1414] [PG 2 Rank 3] Process group watchdog thread terminated with exception: [Rank 3] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=29100340, OpType=GATHER, NumelIn=38016, NumelOut=0, Timeout(ms)=600000) ran for 600051 milliseconds before timing out.
Exception raised from checkTimeout at ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:565 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7f5cf7ba9897 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libc10.so)
frame #1: c10d::ProcessGroupNCCL::WorkNCCL::checkTimeout(std::optional<std::chrono::duration<long, std::ratio<1l, 1000l> > >) + 0x1d2 (0x7f5cf8e82c62 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
frame #2: c10d::ProcessGroupNCCL::watchdogHandler() + 0x1a0 (0x7f5cf8e87a80 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
frame #3: c10d::ProcessGroupNCCL::ncclCommWatchdog() + 0x10c (0x7f5cf8e88dcc in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
frame #4:
[rank1]:[E ProcessGroupNCCL.cpp:563] [Rank 1] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=29100340, OpType=GATHER, NumelIn=38016, NumelOut=0, Timeout(ms)=600000) ran for 600053 milliseconds before timing out.
[rank1]:[E ProcessGroupNCCL.cpp:1537] [PG 2 Rank 1] Timeout at NCCL work: 29100340, last enqueued NCCL work: 29100340, last completed NCCL work: 29100339.
[rank1]:[E ProcessGroupNCCL.cpp:577] [Rank 1] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data.
[rank1]:[E ProcessGroupNCCL.cpp:583] [Rank 1] To avoid data inconsistency, we are taking the entire process down.
[rank1]:[E ProcessGroupNCCL.cpp:1414] [PG 2 Rank 1] Process group watchdog thread terminated with exception: [Rank 1] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=29100340, OpType=GATHER, NumelIn=38016, NumelOut=0, Timeout(ms)=600000) ran for 600053 milliseconds before timing out.
Exception raised from checkTimeout at ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:565 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7f5cf7ba9897 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libc10.so)
frame #1: c10d::ProcessGroupNCCL::WorkNCCL::checkTimeout(std::optional<std::chrono::duration<long, std::ratio<1l, 1000l> > >) + 0x1d2 (0x7f5cf8e82c62 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
frame #2: c10d::ProcessGroupNCCL::watchdogHandler() + 0x1a0 (0x7f5cf8e87a80 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
frame #3: c10d::ProcessGroupNCCL::ncclCommWatchdog() + 0x10c (0x7f5cf8e88dcc in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
frame #4:
[rank0]:[E ProcessGroupNCCL.cpp:563] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=29100340, OpType=GATHER, NumelIn=38016, NumelOut=38016, Timeout(ms)=600000) ran for 600079 milliseconds before timing out.
[rank0]:[E ProcessGroupNCCL.cpp:1537] [PG 2 Rank 0] Timeout at NCCL work: 29100340, last enqueued NCCL work: 29100340, last completed NCCL work: 29100339.
[rank0]:[E ProcessGroupNCCL.cpp:577] [Rank 0] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data.
[rank0]:[E ProcessGroupNCCL.cpp:583] [Rank 0] To avoid data inconsistency, we are taking the entire process down.
[rank0]:[E ProcessGroupNCCL.cpp:1414] [PG 2 Rank 0] Process group watchdog thread terminated with exception: [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=29100340, OpType=GATHER, NumelIn=38016, NumelOut=38016, Timeout(ms)=600000) ran for 600079 milliseconds before timing out.
Exception raised from checkTimeout at ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:565 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7f5cf7ba9897 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libc10.so)
frame #1: c10d::ProcessGroupNCCL::WorkNCCL::checkTimeout(std::optional<std::chrono::duration<long, std::ratio<1l, 1000l> > >) + 0x1d2 (0x7f5cf8e82c62 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
frame #2: c10d::ProcessGroupNCCL::watchdogHandler() + 0x1a0 (0x7f5cf8e87a80 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
frame #3: c10d::ProcessGroupNCCL::ncclCommWatchdog() + 0x10c (0x7f5cf8e88dcc in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
frame #4:
/raid/demo/anaconda3/envs/vllm/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d ' /raid/demo/anaconda3/envs/vllm/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d ' /raid/demo/anaconda3/envs/vllm/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '
ruleGreen
commented
1 month ago same here
hxhcreate
commented
1 month ago same here
Guochry
commented
1 month ago same here
double-vin
commented
1 month ago Once multiple GPU cards are used, this problem will occur.
CREESTL
commented
1 month ago +1
Clint-chan
commented
1 month ago i face a new bug now,
INFO: 61.171.72.231:10464 - "POST /v1/chat/completions HTTP/1.1" 200 OK
INFO 10-16 11:05:06 model_runner_base.py:120] Writing input of failed execution to /tmp/err_execute_model_input_20241016-110506.pkl...
[rank0]:[E1016 11:05:06.676414458 ProcessGroupNCCL.cpp:1515] [PG 3 Rank 0] Process group watchdog thread terminated with exception: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
Exception raised from c10_cuda_check_implementation at ../c10/cuda/CUDAException.cpp:43 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x96 (0x7f87881a0f86 in /raid/demo/anaconda3/envs/vllm_latest/lib/python3.12/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const, char const, unsigned int, std::string const&) + 0x64 (0x7f878814fd10 in /raid/demo/anaconda3/envs/vllm_latest/lib/python3.12/site-packages/torch/lib/libc10.so)
frame #2: c10::cuda::c10_cuda_check_implementation(int, char const, char const, int, bool) + 0x118 (0x7f878827bf08 in /raid/demo/anaconda3/envs/vllm_latest/lib/python3.12/site-packages/torch/lib/libc10_cuda.so)
frame #3: c10d::ProcessGroupNCCL::WorkNCCL::finishedGPUExecutionInternal() const + 0x56 (0x7f87894983e6 in /raid/demo/anaconda3/envs/vllm_latest/lib/python3.12/site-packages/torch/lib/libtorch_cuda.so)
frame #4: c10d::ProcessGroupNCCL::WorkNCCL::isCompleted() + 0xa0 (0x7f878949d600 in /raid/demo/anaconda3/envs/vllm_latest/lib/python3.12/site-packages/torch/lib/libtorch_cuda.so)
frame #5: c10d::ProcessGroupNCCL::watchdogHandler() + 0x1da (0x7f87894a42ba in /raid/demo/anaconda3/envs/vllm_latest/lib/python3.12/site-packages/torch/lib/libtorch_cuda.so)
frame #6: c10d::ProcessGroupNCCL::ncclCommWatchdog() + 0x10c (0x7f87894a66fc in /raid/demo/anaconda3/envs/vllm_latest/lib/python3.12/site-packages/torch/lib/libtorch_cuda.so)
frame #7:
INFO 10-16 11:05:09 logger.py:36] Received request chat-af9601cd9bc24371bb0b11c4a36bc55f: prompt: '<|im_start|>system\n<|im_end|>\n<|im_start|>user\n\n## Role: 专业AI翻译助手\n- description: 你是一个专业的AI翻译助手，精通中文。\n\n## Skills\n1. 精通多种语言的翻译技巧\n2. 准确理解和传达原文含义\n3. 严格遵循输出格式要求\n4. 专业术语和品牌名称的处理能力\n\n## Rules\n1. 仅输出原文和翻译，不添加任何解释或评论。\n2. 严格按照指定的JSON格式提供翻译结果。\n3. 保持专业术语和品牌名称不变，除非有官方的中文译名。\n4. 确保翻译准确传达原文含义，同时保持中文的自然流畅。\n5. 如遇无法翻译的内容，保持原文不变，不做额外说明。\n6. 禁止在输出中包含任何额外的解释、注释或元数据。\n\n## Workflow\n1. 接收翻译任务：目标语言为中文，待翻译文本为"BATERIA RECARGABLE PARA FUENTE DE VOLTAJE UNINTERRUPTIBLE"。\n2. 分析文本，识别专业术语和品牌名称。\n3. 进行翻译，遵循翻译规则。\n4. 按照指定的JSON格式输出结果。\n\n## OutputFormat\n{\n "翻译后": ""\n}\n\n## Init\n你的任务是将给定的文本"BATERIA RECARGABLE PARA FUENTE DE VOLTAJE UNINTERRUPTIBLE"准确翻译成中文。请直接开始翻译工作，只返回要求的JSON格式结果。<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.5, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=7868, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: [151644, 8948, 198, 151645, 198, 151644, 872, 271, 565, 15404, 25, 220, 99878, 15469, 105395, 110498, 198, 12, 4008, 25, 220, 56568, 101909, 104715, 15469, 105395, 110498, 3837, 114806, 104811, 3407, 565, 30240, 198, 16, 13, 10236, 110, 122, 31935, 101312, 102064, 9370, 105395, 102118, 198, 17, 13, 65727, 228, 33956, 115167, 107707, 103283, 109091, 198, 18, 13, 220, 100470, 106466, 66017, 68805, 101882, 198, 19, 13, 220, 99878, 116925, 33108, 100135, 29991, 9370, 54542, 99788, 271, 565, 22847, 198, 16, 13, 220, 99373, 66017, 103283, 33108, 105395, 3837, 16530, 42855, 99885, 104136, 57191, 85641, 8997, 17, 13, 220, 110439, 105146, 9370, 5370, 68805, 99553, 105395, 59151, 8997, 18, 13, 220, 100662, 99878, 116925, 33108, 100135, 29991, 105928, 3837, 106781, 18830, 100777, 9370, 104811, 102610, 13072, 8997, 19, 13, 10236, 94, 106, 32463, 105395, 102188, 107707, 103283, 109091, 3837, 91572, 100662, 104811, 9370, 99795, 110205, 8997, 20, 13, 69372, 99688, 101068, 105395, 104597, 3837, 100662, 103283, 105928, 3837, 109513, 108593, 66394, 8997, 21, 13, 10236, 99, 223, 81433, 18493, 66017, 15946, 102298, 99885, 108593, 9370, 104136, 5373, 25074, 68862, 57191, 23305, 20074, 3407, 565, 60173, 198, 16, 13, 46602, 98, 50009, 105395, 88802, 5122, 100160, 102064, 17714, 104811, 3837, 74193, 105395, 108704, 17714, 63590, 19157, 5863, 74136, 7581, 3494, 50400, 95512, 93777, 3385, 68226, 15204, 40329, 6643, 3221, 48584, 13568, 1, 8997, 17, 13, 58657, 97771, 108704, 3837, 102450, 99878, 116925, 33108, 100135, 29991, 8997, 18, 13, 32181, 249, 22243, 105395, 3837, 106466, 105395, 104190, 8997, 19, 13, 6567, 234, 231, 99331, 105146, 9370, 5370, 68805, 66017, 59151, 3407, 565, 9258, 4061, 198, 515, 220, 330, 105395, 33447, 788, 8389, 630, 565, 15690, 198, 103929, 88802, 20412, 44063, 89012, 22382, 9370, 108704, 63590, 19157, 5863, 74136, 7581, 3494, 50400, 95512, 93777, 3385, 68226, 15204, 40329, 6643, 3221, 48584, 13568, 1, 102188, 105395, 12857, 104811, 1773, 14880, 101041, 55286, 105395, 99257, 3837, 91680, 31526, 101882, 9370, 5370, 68805, 59151, 1773, 151645, 198, 151644, 77091, 198], lora_request: None, prompt_adapter_request: None. INFO 10-16 11:05:10 logger.py:36] Received request chat-8e11f18d93b74c4da90afc8c102132da: prompt: '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nYou need to decompose the user\'s input into "subject" and "intention" in order to accurately figure out what the user\'s input language actually is. \nNotice: the language type user use could be diverse, which can be English, Chinese, Español, Arabic, Japanese, French, and etc.\nMAKE SURE your output is the SAME language as the user\'s input!\nYour output is restricted only to: (Input language) Intention + Subject(short as possible)\nYour output MUST be a valid JSON.\n\nTip: When the user\'s question is directed at you (the language model), you can add an emoji to make it more fun.\n\n\nexample 1:\nUser Input: hi, yesterday i had some burgers.\n{\n "Language Type": "The user\'s input is pure English",\n "Your Reasoning": "The language of my output must be pure English.",\n "Your Output": "sharing yesterday\'s food"\n}\n\nexample 2:\nUser Input: hello\n{\n "Language Type": "The user\'s input is written in pure English",\n "Your Reasoning": "The language of my output must be pure English.",\n "Your Output": "Greeting myself☺️"\n}\n\n\nexample 3:\nUser Input: why mmap file: oom\n{\n "Language Type": "The user\'s input is written in pure English",\n "Your Reasoning": "The language of my output must be pure English.",\n "Your Output": "Asking about the reason for mmap file: oom"\n}\n\n\nexample 4:\nUser Input: www.convinceme.yesterday-you-ate-seafood.tv讲了什么？\n{\n "Language Type": "The user\'s input English-Chinese mixed",\n "Your Reasoning": "The English-part is an URL, the main intention is still written in Chinese, so the language of my output must be using Chinese.",\n "Your Output": "询问网站www.convinceme.yesterday-you-ate-seafood.tv"\n}\n\nexample 5:\nUser Input: why小红的年龄is老than小明？\n{\n "Language Type": "The user\'s input is English-Chinese mixed",\n "Your Reasoning": "The English parts are subjective particles, the main intention is written in Chinese, besides, Chinese occupies a greater "actual meaning" than English, so the language of my output must be using Chinese.",\n "Your Output": "询问小红和小明的年龄"\n}\n\nexample 6:\nUser Input: yo, 你今天咋样？\n{\n "Language Type": "The user\'s input is English-Chinese mixed",\n "Your Reasoning": "The English-part is a subjective particle, the main intention is written in Chinese, so the language of my output must be using Chinese.",\n "Your Output": "查询今日我的状态☺️"\n}\n\nUser Input: \n Fractal Gaming AB\n<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=1.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=100, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: [151644, 8948, 198, 2610, 525, 264, 10950, 17847, 13, 151645, 198, 151644, 872, 198, 2610, 1184, 311, 28502, 2900, 279, 1196, 594, 1946, 1119, 330, 11501, 1, 323, 330, 396, 2939, 1, 304, 1973, 311, 29257, 7071, 700, 1128, 279, 1196, 594, 1946, 4128, 3520, 374, 13, 715, 34193, 25, 279, 4128, 943, 1196, 990, 1410, 387, 16807, 11, 892, 646, 387, 6364, 11, 8453, 11, 142658, 11, 34117, 11, 10769, 11, 8585, 11, 323, 4992, 624, 47082, 328, 4521, 697, 2550, 374, 279, 83490, 4128, 438, 279, 1196, 594, 1946, 4894, 7771, 2550, 374, 21739, 1172, 311, 25, 320, 2505, 4128, 8, 1333, 2939, 488, 17450, 37890, 438, 3204, 340, 7771, 2550, 27732, 387, 264, 2697, 4718, 382, 16011, 25, 3197, 279, 1196, 594, 3405, 374, 15540, 518, 498, 320, 1782, 4128, 1614, 701, 498, 646, 912, 458, 42365, 311, 1281, 432, 803, 2464, 4192, 8687, 220, 16, 510, 1474, 5571, 25, 15588, 11, 13671, 600, 1030, 1045, 62352, 624, 515, 220, 330, 13806, 3990, 788, 330, 785, 1196, 594, 1946, 374, 10526, 6364, 756, 220, 330, 7771, 26759, 287, 788, 330, 785, 4128, 315, 847, 2550, 1969, 387, 10526, 6364, 10346, 220, 330, 7771, 9258, 788, 330, 83646, 13671, 594, 3607, 698, 630, 8687, 220, 17, 510, 1474, 5571, 25, 23811, 198, 515, 220, 330, 13806, 3990, 788, 330, 785, 1196, 594, 1946, 374, 5326, 304, 10526, 6364, 756, 220, 330, 7771, 26759, 287, 788, 330, 785, 4128, 315, 847, 2550, 1969, 387, 10526, 6364, 10346, 220, 330, 7771, 9258, 788, 330, 38, 43632, 7037, 144346, 30543, 698, 3733, 8687, 220, 18, 510, 1474, 5571, 25, 3170, 70866, 1034, 25, 297, 316, 198, 515, 220, 330, 13806, 3990, 788, 330, 785, 1196, 594, 1946, 374, 5326, 304, 10526, 6364, 756, 220, 330, 7771, 26759, 287, 788, 330, 785, 4128, 315, 847, 2550, 1969, 387, 10526, 6364, 10346, 220, 330, 7771, 9258, 788, 330, 2121, 10566, 911, 279, 2874, 369, 70866, 1034, 25, 297, 316, 698, 3733, 8687, 220, 19, 510, 1474, 5571, 25, 8438, 25961, 2840, 3894, 2384, 11282, 45417, 12, 349, 7806, 64, 13915, 14485, 99526, 34187, 99245, 94432, 515, 220, 330, 13806, 3990, 788, 330, 785, 1196, 594, 1946, 6364, 29553, 7346, 9519, 756, 220, 330, 7771, 26759, 287, 788, 330, 785, 6364, 28037, 374, 458, 5548, 11, 279, 1887, 14602, 374, 2058, 5326, 304, 8453, 11, 773, 279, 4128, 315, 847, 2550, 1969, 387, 1667, 8453, 10346, 220, 330, 7771, 9258, 788, 330, 105396, 100010, 2136, 25961, 2840, 3894, 2384, 11282, 45417, 12, 349, 7806, 64, 13915, 14485, 698, 630, 8687, 220, 20, 510, 1474, 5571, 25, 3170, 30709, 99425, 9370, 102185, 285, 91777, 53795, 30709, 30858, 94432, 515, 220, 330, 13806, 3990, 788, 330, 785, 1196, 594, 1946, 374, 6364, 29553, 7346, 9519, 756, 220, 330, 7771, 26759, 287, 788, 330, 785, 6364, 5479, 525, 43022, 18730, 11, 279, 1887, 14602, 374, 5326, 304, 8453, 11, 27758, 11, 8453, 75754, 264, 7046, 330, 11944, 7290, 1, 1091, 6364, 11, 773, 279, 4128, 315, 847, 2550, 1969, 387, 1667, 8453, 10346, 220, 330, 7771, 9258, 788, 330, 105396, 30709, 99425, 33108, 30709, 30858, 9370, 102185, 698, 630, 8687, 220, 21, 510, 1474, 5571, 25, 29496, 11, 220, 56568, 100644, 108806, 90885, 94432, 515, 220, 330, 13806, 3990, 788, 330, 785, 1196, 594, 1946, 374, 6364, 29553, 7346, 9519, 756, 220, 330, 7771, 26759, 287, 788, 330, 785, 6364, 28037, 374, 264, 43022, 18790, 11, 279, 1887, 14602, 374, 5326, 304, 8453, 11, 773, 279, 4128, 315, 847, 2550, 1969, 387, 1667, 8453, 10346, 220, 330, 7771, 9258, 788, 330, 51154, 102242, 97611, 44091, 144346, 30543, 698, 630, 1474, 5571, 25, 715, 2869, 80148, 30462, 14137, 198, 151645, 198, 151644, 77091, 198], lora_request: None, prompt_adapter_request: None. INFO 10-16 11:05:10 logger.py:36] Received request chat-7a3f5c6924f44630ac4b60b68ddc61d7: prompt: '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nYou need to decompose the user\'s input into "subject" and "intention" in order to accurately figure out what the user\'s input language actually is. \nNotice: the language type user use could be diverse, which can be English, Chinese, Español, Arabic, Japanese, French, and etc.\nMAKE SURE your output is the SAME language as the user\'s input!\nYour output is restricted only to: (Input language) Intention + Subject(short as possible)\nYour output MUST be a valid JSON.\n\nTip: When the user\'s question is directed at you (the language model), you can add an emoji to make it more fun.\n\n\nexample 1:\nUser Input: hi, yesterday i had some burgers.\n{\n "Language Type": "The user\'s input is pure English",\n "Your Reasoning": "The language of my output must be pure English.",\n "Your Output": "sharing yesterday\'s food"\n}\n\nexample 2:\nUser Input: hello\n{\n "Language Type": "The user\'s input is written in pure English",\n "Your Reasoning": "The language of my output must be pure English.",\n "Your Output": "Greeting myself☺️"\n}\n\n\nexample 3:\nUser Input: why mmap file: oom\n{\n "Language Type": "The user\'s input is written in pure English",\n "Your Reasoning": "The language of my output must be pure English.",\n "Your Output": "Asking about the reason for mmap file: oom"\n}\n\n\nexample 4:\nUser Input: www.convinceme.yesterday-you-ate-seafood.tv讲了什么？\n{\n "Language Type": "The user\'s input English-Chinese mixed",\n "Your Reasoning": "The English-part is an URL, the main intention is still written in Chinese, so the language of my output must be using Chinese.",\n "Your Output": "询问网站www.convinceme.yesterday-you-ate-seafood.tv"\n}\n\nexample 5:\nUser Input: why小红的年龄is老than小明？\n{\n "Language Type": "The user\'s input is English-Chinese mixed",\n "Your Reasoning": "The English parts are subjective particles, the main intention is written in Chinese, besides, Chinese occupies a greater "actual meaning" than English, so the language of my output must be using Chinese.",\n "Your Output": "询问小红和小明的年龄"\n}\n\nexample 6:\nUser Input: yo, 你今天咋样？\n{\n "Language Type": "The user\'s input is English-Chinese mixed",\n "Your Reasoning": "The English-part is a subjective particle, the main intention is written in Chinese, so the language of my output must be using Chinese.",\n "Your Output": "查询今日我的状态☺️"\n}\n\nUser Input: \n Fractal Gaming AB\n<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=1.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=100, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: [151644, 8948, 198, 2610, 525, 264, 10950, 17847, 13, 151645, 198, 151644, 872, 198, 2610, 1184, 311, 28502, 2900, 279, 1196, 594, 1946, 1119, 330, 11501, 1, 323, 330, 396, 2939, 1, 304, 1973, 311, 29257, 7071, 700, 1128, 279, 1196, 594, 1946, 4128, 3520, 374, 13, 715, 34193, 25, 279, 4128, 943, 1196, 990, 1410, 387, 16807, 11, 892, 646, 387, 6364, 11, 8453, 11, 142658, 11, 34117, 11, 10769, 11, 8585, 11, 323, 4992, 624, 47082, 328, 4521, 697, 2550, 374, 279, 83490, 4128, 438, 279, 1196, 594, 1946, 4894, 7771, 2550, 374, 21739, 1172, 311, 25, 320, 2505, 4128, 8, 1333, 2939, 488, 17450, 37890, 438, 3204, 340, 7771, 2550, 27732, 387, 264, 2697, 4718, 382, 16011, 25, 3197, 279, 1196, 594, 3405, 374, 15540, 518, 498, 320, 1782, 4128, 1614, 701, 498, 646, 912, 458, 42365, 311, 1281, 432, 803, 2464, 4192, 8687, 220, 16, 510, 1474, 5571, 25, 15588, 11, 13671, 600, 1030, 1045, 62352, 624, 515, 220, 330, 13806, 3990, 788, 330, 785, 1196, 594, 1946, 374, 10526, 6364, 756, 220, 330, 7771, 26759, 287, 788, 330, 785, 4128, 315, 847, 2550, 1969, 387, 10526, 6364, 10346, 220, 330, 7771, 9258, 788, 330, 83646, 13671, 594, 3607, 698, 630, 8687, 220, 17, 510, 1474, 5571, 25, 23811, 198, 515, 220, 330, 13806, 3990, 788, 330, 785, 1196, 594, 1946, 374, 5326, 304, 10526, 6364, 756, 220, 330, 7771, 26759, 287, 788, 330, 785, 4128, 315, 847, 2550, 1969, 387, 10526, 6364, 10346, 220, 330, 7771, 9258, 788, 330, 38, 43632, 7037, 144346, 30543, 698, 3733, 8687, 220, 18, 510, 1474, 5571, 25, 3170, 70866, 1034, 25, 297, 316, 198, 515, 220, 330, 13806, 3990, 788, 330, 785, 1196, 594, 1946, 374, 5326, 304, 10526, 6364, 756, 220, 330, 7771, 26759, 287, 788, 330, 785, 4128, 315, 847, 2550, 1969, 387, 10526, 6364, 10346, 220, 330, 7771, 9258, 788, 330, 2121, 10566, 911, 279, 2874, 369, 70866, 1034, 25, 297, 316, 698, 3733, 8687, 220, 19, 510, 1474, 5571, 25, 8438, 25961, 2840, 3894, 2384, 11282, 45417, 12, 349, 7806, 64, 13915, 14485, 99526, 34187, 99245, 94432, 515, 220, 330, 13806, 3990, 788, 330, 785, 1196, 594, 1946, 6364, 29553, 7346, 9519, 756, 220, 330, 7771, 26759, 287, 788, 330, 785, 6364, 28037, 374, 458, 5548, 11, 279, 1887, 14602, 374, 2058, 5326, 304, 8453, 11, 773, 279, 4128, 315, 847, 2550, 1969, 387, 1667, 8453, 10346, 220, 330, 7771, 9258, 788, 330, 105396, 100010, 2136, 25961, 2840, 3894, 2384, 11282, 45417, 12, 349, 7806, 64, 13915, 14485, 698, 630, 8687, 220, 20, 510, 1474, 5571, 25, 3170, 30709, 99425, 9370, 102185, 285, 91777, 53795, 30709, 30858, 94432, 515, 220, 330, 13806, 3990, 788, 330, 785, 1196, 594, 1946, 374, 6364, 29553, 7346, 9519, 756, 220, 330, 7771, 26759, 287, 788, 330, 785, 6364, 5479, 525, 43022, 18730, 11, 279, 1887, 14602, 374, 5326, 304, 8453, 11, 27758, 11, 8453, 75754, 264, 7046, 330, 11944, 7290, 1, 1091, 6364, 11, 773, 279, 4128, 315, 847, 2550, 1969, 387, 1667, 8453, 10346, 220, 330, 7771, 9258, 788, 330, 105396, 30709, 99425, 33108, 30709, 30858, 9370, 102185, 698, 630, 8687, 220, 21, 510, 1474, 5571, 25, 29496, 11, 220, 56568, 100644, 108806, 90885, 94432, 515, 220, 330, 13806, 3990, 788, 330, 785, 1196, 594, 1946, 374, 6364, 29553, 7346, 9519, 756, 220, 330, 7771, 26759, 287, 788, 330, 785, 6364, 28037, 374, 264, 43022, 18790, 11, 279, 1887, 14602, 374, 5326, 304, 8453, 11, 773, 279, 4128, 315, 847, 2550, 1969, 387, 1667, 8453, 10346, 220, 330, 7771, 9258, 788, 330, 51154, 102242, 97611, 44091, 144346, 30543, 698, 630, 1474, 5571, 25, 715, 2869, 80148, 30462, 14137, 198, 151645, 198, 151644, 77091, 198], lora_request: None, prompt_adapter_request: None. INFO 10-16 11:05:10 logger.py:36] Received request chat-33cd06cd1f474b76a0bde5ecc2beb72a: prompt: '<|im_start|>system\n假如你是一个贸易情报助手，根据给定的词语给出十个英文近义词\n示例：\n\nquestion：\nled\n\nanswer：\n["Light Emitting Diode", "Indicator Light", "Backlight", "Display Screen", "Lamp", "Bulb", "Illuminator", "Glow Stick", "Neon Sign", "Fluorescent Tube"]\n\nquestion:\napple\n\nanswer:\n["Apple", "Fruit", "Malus domestica", "Red Delicious Apple", "Granny Smith Apple", "Gala Apple", "Fuji Apple", "Honeycrisp Apple", "Cripps Pink Apple", "Pink Lady Apple"]\n\n请严格按照这样的数组格式输出，切勿输出其他内容\n请注意，将用户输入的词语首字母大写不能被认定为，不要输出这样的内容\nValve不是valve的近义词\nApple不是apple的近义词\nFruit不是fruit的近义词<|im_end|>\n<|im_start|>user\n Fractal Gaming AB\n\n请生成这样格式的输出：\n其中数组中每个元素为近义词\n["","","","","","","","","",""]\n类似于\n["Apple", "Fruit", "Malus domestica", "Red Delicious Apple", "Granny Smith Apple", "Gala Apple", "Fuji Apple", "Honeycrisp Apple", "Cripps Pink Apple", "Pink Lady Apple"]\n注意，这些近义词全是英语\n注意，将问题的首字母进行大写不能为认定为近义词\n注意，只要返回这个数组即可，其他内容不要返回<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.5, frequency_penalty=0.5, repetition_penalty=1.0, temperature=0.9, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=7859, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: [151644, 8948, 198, 105812, 56568, 101909, 100726, 108442, 110498, 3837, 100345, 89012, 22382, 9370, 113042, 107485, 110661, 105205, 59258, 64559, 99689, 198, 19793, 26355, 48443, 7841, 28311, 832, 271, 9217, 28311, 1183, 13911, 61537, 1280, 7767, 534, 497, 330, 19523, 8658, 497, 330, 3707, 4145, 497, 330, 7020, 13948, 497, 330, 43, 1121, 497, 330, 33, 360, 65, 497, 330, 40, 5448, 17272, 497, 330, 38, 10303, 46461, 497, 330, 8813, 263, 7075, 497, 330, 3882, 84, 4589, 1168, 29024, 25912, 7841, 510, 22377, 271, 9217, 510, 1183, 26567, 497, 330, 37, 21026, 497, 330, 29600, 355, 73422, 3001, 497, 330, 6033, 84688, 8162, 497, 330, 6464, 12888, 9082, 8162, 497, 330, 38, 6053, 8162, 497, 330, 76745, 7754, 8162, 497, 330, 39, 2534, 5082, 13090, 8162, 497, 330, 34, 54689, 82, 26119, 8162, 497, 330, 71352, 20621, 8162, 25912, 14880, 110439, 101893, 69824, 68805, 66017, 3837, 99322, 108710, 66017, 92894, 43815, 198, 118271, 3837, 44063, 20002, 31196, 9370, 113042, 59975, 110788, 26288, 61443, 53153, 99250, 104585, 17714, 3837, 100148, 66017, 101893, 43815, 198, 2208, 586, 99520, 831, 586, 9370, 59258, 64559, 99689, 198, 26567, 99520, 22377, 9370, 59258, 64559, 99689, 198, 37, 21026, 99520, 35598, 9370, 59258, 64559, 99689, 151645, 198, 151644, 872, 198, 2869, 80148, 30462, 14137, 271, 14880, 43959, 99654, 68805, 9370, 66017, 28311, 90919, 69824, 15946, 103991, 102268, 17714, 59258, 64559, 99689, 198, 1183, 59859, 59859, 59859, 59859, 2198, 7026, 113080, 198, 1183, 26567, 497, 330, 37, 21026, 497, 330, 29600, 355, 73422, 3001, 497, 330, 6033, 84688, 8162, 497, 330, 6464, 12888, 9082, 8162, 497, 330, 38, 6053, 8162, 497, 330, 76745, 7754, 8162, 497, 330, 39, 2534, 5082, 13090, 8162, 497, 330, 34, 54689, 82, 26119, 8162, 497, 330, 71352, 20621, 8162, 7026, 60533, 3837, 100001, 59258, 64559, 99689, 108743, 104105, 198, 60533, 3837, 44063, 86119, 9370, 59975, 110788, 71817, 26288, 61443, 53153, 17714, 104585, 17714, 59258, 64559, 99689, 198, 60533, 3837, 100671, 31526, 99487, 69824, 104180, 3837, 92894, 43815, 100148, 31526, 151645, 198, 151644, 77091, 198], lora_request: None, prompt_adapter_request: None. INFO: 61.171.72.231:17908 - "POST /v1/chat/completions HTTP/1.1" 200 OK INFO 10-16 11:05:10 logger.py:36] Received request chat-8b4bcd8b55c348ceac242190083e0c6c: prompt: '<|im_start|>system\n根据用户输入的商品关键字，生成一个包含JSON对象列表的字符串。每个对象的键是一个行业类别，值是一个五个商品类别的列表。确保有三个不同的行业类别，每个类别包含五个商品类别。每个行业关键字对应的产品不能出现重复，也不能和用户的输入重复，如果出现重复，可以再生成一个产品关键字替代一下。\n请注意key字段对应的值必须是中文，value字段对应的值必须是英文。\n请不要返回代码块\n请严格按照下面的格式输出，不要输出其余内容，谢谢你\n以下是JSON格式示例：\n[\n {\n "key": "手机行业",\n "value": ["Computer Hardware", "IT Equipment", "Electronic Components", "Software Solutions", "Networking Devices"]\n },\n {\n "key": "消费电子行业",\n "value": ["Computer Hardware", "IT Equipment", "Electronic Components", "Software Solutions", "Networking Devices"]\n },\n {\n "key": "科技设备行业",\n "value": ["Computer Hardware", "IT Equipment", "Electronic Components", "Software Solutions", "Networking Devices"]\n }\n]<|im_end|>\n<|im_start|>user\n Fractal Gaming AB<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.7, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=7947, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: [151644, 8948, 198, 100345, 20002, 31196, 109348, 118294, 3837, 43959, 46944, 102298, 5370, 64429, 44177, 9370, 66558, 1773, 103991, 64429, 9370, 60949, 101909, 99717, 107975, 3837, 25511, 101909, 105220, 45943, 21515, 102657, 44177, 1773, 103944, 18830, 101124, 101970, 99717, 107975, 3837, 103991, 107975, 102298, 105220, 45943, 107975, 1773, 103991, 99717, 118294, 103124, 104310, 53153, 100347, 105444, 3837, 105827, 33108, 107494, 31196, 105444, 3837, 62244, 100347, 105444, 3837, 73670, 87256, 43959, 46944, 82700, 118294, 105598, 100158, 8997, 118271, 792, 44931, 110019, 25511, 100645, 20412, 104811, 3837, 957, 44931, 110019, 25511, 100645, 20412, 105205, 8997, 14880, 100148, 31526, 46100, 99922, 198, 14880, 110439, 100431, 9370, 68805, 66017, 3837, 100148, 66017, 106177, 43815, 3837, 116642, 198, 114566, 5370, 68805, 19793, 26355, 28311, 9640, 220, 341, 262, 330, 792, 788, 330, 58405, 99717, 756, 262, 330, 957, 788, 4383, 37332, 36765, 497, 330, 952, 20236, 497, 330, 89643, 34085, 497, 330, 19250, 22676, 497, 330, 78007, 40377, 7026, 220, 1153, 220, 341, 262, 330, 792, 788, 330, 100030, 100382, 99717, 756, 262, 330, 957, 788, 4383, 37332, 36765, 497, 330, 952, 20236, 497, 330, 89643, 34085, 497, 330, 19250, 22676, 497, 330, 78007, 40377, 7026, 220, 1153, 220, 341, 262, 330, 792, 788, 330, 99602, 101044, 99717, 756, 262, 330, 957, 788, 4383, 37332, 36765, 497, 330, 952, 20236, 497, 330, 89643, 34085, 497, 330, 19250, 22676, 497, 330, 78007, 40377, 7026, 220, 456, 60, 151645, 198, 151644, 872, 198, 2869, 80148, 30462, 14137, 151645, 198, 151644, 77091, 198], lora_request: None, prompt_adapter_request: None. INFO: 61.171.72.231:28573 - "POST /v1/chat/completions HTTP/1.1" 200 OK INFO 10-16 11:05:11 logger.py:36] Received request chat-122f7dc0a4a04df6bbdb7f08a718ffbd: prompt: '<|im_start|>system\n<|im_end|>\n<|im_start|>user\n\n## Role: 专业AI翻译助手\n- description: 你是一个专业的AI翻译助手，精通中文。\n\n## Skills\n1. 精通多种语言的翻译技巧\n2. 准确理解和传达原文含义\n3. 严格遵循输出格式要求\n4. 专业术语和品牌名称的处理能力\n\n## Rules\n1. 仅输出原文和翻译，不添加任何解释或评论。\n2. 严格按照指定的JSON格式提供翻译结果。\n3. 保持专业术语和品牌名称不变，除非有官方的中文译名。\n4. 确保翻译准确传达原文含义，同时保持中文的自然流畅。\n5. 如遇无法翻译的内容，保持原文不变，不做额外说明。\n6. 禁止在输出中包含任何额外的解释、注释或元数据。\n\n## Workflow\n1. 接收翻译任务：目标语言为中文，待翻译文本为"CNNS SAPROPEL ORGANIC FERTILIZER, IN POWDER AND GRANULAR FORM. RATIO ORGANIC MATTER 60%; TOTAL PROTEIN 2.5; HUMIC ACID 1.8 C N 12; PHH20 5;\xa0HUMIDITY 30%.\xa0PACKAGING 500 KG BAG (GOODS WITHOUT LABELS) 100% NEW"。\n2. 分析文本，识别专业术语和品牌名称。\n3. 进行翻译，遵循翻译规则。\n4. 按照指定的JSON格式输出结果。\n\n## OutputFormat\n{\n "翻译后": ""\n}\n\n## Init\n你的任务是将给定的文本"CNNS SAPROPEL ORGANIC FERTILIZER, IN POWDER AND GRANULAR FORM. RATIO ORGANIC MATTER 60%; TOTAL PROTEIN 2.5; HUMIC ACID 1.8 C N 12; PHH20 5;\xa0HUMIDITY 30%.\xa0PACKAGING 500 KG BAG (GOODS WITHOUT LABELS) 100% NEW"准确翻译成中文。请直接开始翻译工作，只返回要求的JSON格式结果。<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.5, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=7706, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: [151644, 8948, 198, 151645, 198, 151644, 872, 271, 565, 15404, 25, 220, 99878, 15469, 105395, 110498, 198, 12, 4008, 25, 220, 56568, 101909, 104715, 15469, 105395, 110498, 3837, 114806, 104811, 3407, 565, 30240, 198, 16, 13, 10236, 110, 122, 31935, 101312, 102064, 9370, 105395, 102118, 198, 17, 13, 65727, 228, 33956, 115167, 107707, 103283, 109091, 198, 18, 13, 220, 100470, 106466, 66017, 68805, 101882, 198, 19, 13, 220, 99878, 116925, 33108, 100135, 29991, 9370, 54542, 99788, 271, 565, 22847, 198, 16, 13, 220, 99373, 66017, 103283, 33108, 105395, 3837, 16530, 42855, 99885, 104136, 57191, 85641, 8997, 17, 13, 220, 110439, 105146, 9370, 5370, 68805, 99553, 105395, 59151, 8997, 18, 13, 220, 100662, 99878, 116925, 33108, 100135, 29991, 105928, 3837, 106781, 18830, 100777, 9370, 104811, 102610, 13072, 8997, 19, 13, 10236, 94, 106, 32463, 105395, 102188, 107707, 103283, 109091, 3837, 91572, 100662, 104811, 9370, 99795, 110205, 8997, 20, 13, 69372, 99688, 101068, 105395, 104597, 3837, 100662, 103283, 105928, 3837, 109513, 108593, 66394, 8997, 21, 13, 10236, 99, 223, 81433, 18493, 66017, 15946, 102298, 99885, 108593, 9370, 104136, 5373, 25074, 68862, 57191, 23305, 20074, 3407, 565, 60173, 198, 16, 13, 46602, 98, 50009, 105395, 88802, 5122, 100160, 102064, 17714, 104811, 3837, 74193, 105395, 108704, 17714, 1, 28668, 2448, 36221, 1285, 1740, 43, 2726, 58487, 1317, 434, 3399, 1715, 78141, 11, 1964, 69793, 11391, 3567, 14773, 1093, 7081, 27824, 13, 431, 54838, 2726, 58487, 1317, 24795, 4198, 220, 21, 15, 16175, 46962, 5308, 2446, 687, 220, 17, 13, 20, 26, 472, 2794, 1317, 10584, 915, 220, 16, 13, 23, 356, 451, 220, 16, 17, 26, 14659, 39, 17, 15, 220, 20, 26, 4102, 39, 2794, 915, 3333, 220, 18, 15, 14360, 4102, 17279, 79606, 220, 20, 15, 15, 70087, 425, 1890, 320, 69733, 50, 6007, 56874, 50, 8, 220, 16, 15, 15, 4, 16165, 1, 8997, 17, 13, 58657, 97771, 108704, 3837, 102450, 99878, 116925, 33108, 100135, 29991, 8997, 18, 13, 32181, 249, 22243, 105395, 3837, 106466, 105395, 104190, 8997, 19, 13, 6567, 234, 231, 99331, 105146, 9370, 5370, 68805, 66017, 59151, 3407, 565, 9258, 4061, 198, 515, 220, 330, 105395, 33447, 788, 8389, 630, 565, 15690, 198, 103929, 88802, 20412, 44063, 89012, 22382, 9370, 108704, 1, 28668, 2448, 36221, 1285, 1740, 43, 2726, 58487, 1317, 434, 3399, 1715, 78141, 11, 1964, 69793, 11391, 3567, 14773, 1093, 7081, 27824, 13, 431, 54838, 2726, 58487, 1317, 24795, 4198, 220, 21, 15, 16175, 46962, 5308, 2446, 687, 220, 17, 13, 20, 26, 472, 2794, 1317, 10584, 915, 220, 16, 13, 23, 356, 451, 220, 16, 17, 26, 14659, 39, 17, 15, 220, 20, 26, 4102, 39, 2794, 915, 3333, 220, 18, 15, 14360, 4102, 17279, 79606, 220, 20, 15, 15, 70087, 425, 1890, 320, 69733, 50, 6007, 56874, 50, 8, 220, 16, 15, 15, 4, 16165, 1, 102188, 105395, 12857, 104811, 1773, 14880, 101041, 55286, 105395, 99257, 3837, 91680, 31526, 101882, 9370, 5370, 68805, 59151, 1773, 151645, 198, 151644, 77091, 198], lora_request: None, prompt_adapter_request: None. INFO 10-16 11:05:11 logger.py:36] Received request chat-a0f6ea73195141e2a4e052f8fcad3161: prompt: '<|im_start|>system\n<|im_end|>\n<|im_start|>user\n\n## Role: 专业AI翻译助手\n- description: 你是一个专业的AI翻译助手，精通中文。\n\n## Skills\n1. 精通多种语言的翻译技巧\n2. 准确理解和传达原文含义\n3. 严格遵循输出格式要求\n4. 专业术语和品牌名称的处理能力\n\n## Rules\n1. 仅输出原文和翻译，不添加任何解释或评论。\n2. 严格按照指定的JSON格式提供翻译结果。\n3. 保持专业术语和品牌名称不变，除非有官方的中文译名。\n4. 确保翻译准确传达原文含义，同时保持中文的自然流畅。\n5. 如遇无法翻译的内容，保持原文不变，不做额外说明。\n6. 禁止在输出中包含任何额外的解释、注释或元数据。\n\n## Workflow\n1. 接收翻译任务：目标语言为中文，待翻译文本为"FROZEN CORN ON COB (1X12PKTS., 12.7KG EA"。\n2. 分析文本，识别专业术语和品牌名称。\n3. 进行翻译，遵循翻译规则。\n4. 按照指定的JSON格式输出结果。\n\n## OutputFormat\n{\n "翻译后": ""\n}\n\n## Init\n你的任务是将给定的文本"FROZEN CORN ON COB (1X12PKTS., 12.7KG EA"准确翻译成中文。请直接开始翻译工作，只返回要求的JSON格式结果。<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.5, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=7854, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: [151644, 8948, 198, 151645, 198, 151644, 872, 271, 565, 15404, 25, 220, 99878, 15469, 105395, 110498, 198, 12, 4008, 25, 220, 56568, 101909, 104715, 15469, 105395, 110498, 3837, 114806, 104811, 3407, 565, 30240, 198, 16, 13, 10236, 110, 122, 31935, 101312, 102064, 9370, 105395, 102118, 198, 17, 13, 65727, 228, 33956, 115167, 107707, 103283, 109091, 198, 18, 13, 220, 100470, 106466, 66017, 68805, 101882, 198, 19, 13, 220, 99878, 116925, 33108, 100135, 29991, 9370, 54542, 99788, 271, 565, 22847, 198, 16, 13, 220, 99373, 66017, 103283, 33108, 105395, 3837, 16530, 42855, 99885, 104136, 57191, 85641, 8997, 17, 13, 220, 110439, 105146, 9370, 5370, 68805, 99553, 105395, 59151, 8997, 18, 13, 220, 100662, 99878, 116925, 33108, 100135, 29991, 105928, 3837, 106781, 18830, 100777, 9370, 104811, 102610, 13072, 8997, 19, 13, 10236, 94, 106, 32463, 105395, 102188, 107707, 103283, 109091, 3837, 91572, 100662, 104811, 9370, 99795, 110205, 8997, 20, 13, 69372, 99688, 101068, 105395, 104597, 3837, 100662, 103283, 105928, 3837, 109513, 108593, 66394, 8997, 21, 13, 10236, 99, 223, 81433, 18493, 66017, 15946, 102298, 99885, 108593, 9370, 104136, 5373, 25074, 68862, 57191, 23305, 20074, 3407, 565, 60173, 198, 16, 13, 46602, 98, 50009, 105395, 88802, 5122, 100160, 102064, 17714, 104811, 3837, 74193, 105395, 108704, 17714, 86555, 1285, 57, 953, 26465, 45, 6197, 7284, 33, 320, 16, 55, 16, 17, 22242, 9951, 2572, 220, 16, 17, 13, 22, 42916, 38362, 1, 8997, 17, 13, 58657, 97771, 108704, 3837, 102450, 99878, 116925, 33108, 100135, 29991, 8997, 18, 13, 32181, 249, 22243, 105395, 3837, 106466, 105395, 104190, 8997, 19, 13, 6567, 234, 231, 99331, 105146, 9370, 5370, 68805, 66017, 59151, 3407, 565, 9258, 4061, 198, 515, 220, 330, 105395, 33447, 788, 8389, 630, 565, 15690, 198, 103929, 88802, 20412, 44063, 89012, 22382, 9370, 108704, 86555, 1285, 57, 953, 26465, 45, 6197, 7284, 33, 320, 16, 55, 16, 17, 22242, 9951, 2572, 220, 16, 17, 13, 22, 42916, 38362, 1, 102188, 105395, 12857, 104811, 1773, 14880, 101041, 55286, 105395, 99257, 3837, 91680, 31526, 101882, 9370, 5370, 68805, 59151, 1773, 151645, 198, 151644, 77091, 198], lora_request: None, prompt_adapter_request: None. INFO 10-16 11:05:14 logger.py:36] Received request chat-ce259a293f4947c690b5d0fa7a3746e8: prompt: '<|im_start|>system\n<|im_end|>\n<|im_start|>user\n\n## Role: 专业AI翻译助手\n- description: 你是一个专业的AI翻译助手，精通中文。\n\n## Skills\n1. 精通多种语言的翻译技巧\n2. 准确理解和传达原文含义\n3. 严格遵循输出格式要求\n4. 专业术语和品牌名称的处理能力\n\n## Rules\n1. 仅输出原文和翻译，不添加任何解释或评论。\n2. 严格按照指定的JSON格式提供翻译结果。\n3. 保持专业术语和品牌名称不变，除非有官方的中文译名。\n4. 确保翻译准确传达原文含义，同时保持中文的自然流畅。\n5. 如遇无法翻译的内容，保持原文不变，不做额外说明。\n6. 禁止在输出中包含任何额外的解释、注释或元数据。\n\n## Workflow\n1. 接收翻译任务：目标语言为中文，待翻译文本为"STAINLESS STEEL SCREEN 60 MESH 1.6(FOR INDONESIA)\n"。\n2. 分析文本，识别专业术语和品牌名称。\n3. 进行翻译，遵循翻译规则。\n4. 按照指定的JSON格式输出结果。\n\n## OutputFormat\n{\n "翻译后": ""\n}\n\n## Init\n你的任务是将给定的文本"STAINLESS STEEL SCREEN 60 MESH 1.6(FOR INDONESIA)\n"准确翻译成中文。请直接开始翻译工作，只返回要求的JSON格式结果。<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.5, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=7858, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: [151644, 8948, 198, 151645, 198, 151644, 872, 271, 565, 15404, 25, 220, 99878, 15469, 105395, 110498, 198, 12, 4008, 25, 220, 56568, 101909, 104715, 15469, 105395, 110498, 3837, 114806, 104811, 3407, 565, 30240, 198, 16, 13, 10236, 110, 122, 31935, 101312, 102064, 9370, 105395, 102118, 198, 17, 13, 65727, 228, 33956, 115167, 107707, 103283, 109091, 198, 18, 13, 220, 100470, 106466, 66017, 68805, 101882, 198, 19, 13, 220, 99878, 116925, 33108, 100135, 29991, 9370, 54542, 99788, 271, 565, 22847, 198, 16, 13, 220, 99373, 66017, 103283, 33108, 105395, 3837, 16530, 42855, 99885, 104136, 57191, 85641, 8997, 17, 13, 220, 110439, 105146, 9370, 5370, 68805, 99553, 105395, 59151, 8997, 18, 13, 220, 100662, 99878, 116925, 33108, 100135, 29991, 105928, 3837, 106781, 18830, 100777, 9370, 104811, 102610, 13072, 8997, 19, 13, 10236, 94, 106, 32463, 105395, 102188, 107707, 103283, 109091, 3837, 91572, 100662, 104811, 9370, 99795, 110205, 8997, 20, 13, 69372, 99688, 101068, 105395, 104597, 3837, 100662, 103283, 105928, 3837, 109513, 108593, 66394, 8997, 21, 13, 10236, 99, 223, 81433, 18493, 66017, 15946, 102298, 99885, 108593, 9370, 104136, 5373, 25074, 68862, 57191, 23305, 20074, 3407, 565, 60173, 198, 16, 13, 46602, 98, 50009, 105395, 88802, 5122, 100160, 102064, 17714, 104811, 3837, 74193, 105395, 108704, 17714, 1, 784, 6836, 37773, 26499, 2749, 35073, 220, 21, 15, 386, 40004, 353, 16, 13, 21, 7832, 868, 19317, 60289, 5863, 340, 1, 8997, 17, 13, 58657, 97771, 108704, 3837, 102450, 99878, 116925, 33108, 100135, 29991, 8997, 18, 13, 32181, 249, 22243, 105395, 3837, 106466, 105395, 104190, 8997, 19, 13, 6567, 234, 231, 99331, 105146, 9370, 5370, 68805, 66017, 59151, 3407, 565, 9258, 4061, 198, 515, 220, 330, 105395, 33447, 788, 8389, 630, 565, 15690, 198, 103929, 88802, 20412, 44063, 89012, 22382, 9370, 108704, 1, 784, 6836, 37773, 26499, 2749, 35073, 220, 21, 15, 386, 40004, 353, 16, 13, 21, 7832, 868, 19317, 60289, 5863, 340, 1, 102188, 105395, 12857, 104811, 1773, 14880, 101041, 55286, 105395, 99257, 3837, 91680, 31526, 101882, 9370, 5370, 68805, 59151, 1773, 151645, 198, 151644, 77091, 198], lora_request: None, prompt_adapter_request: None. INFO 10-16 11:05:16 logger.py:36] Received request chat-3a3770ba57c2481886726891062170d4: prompt: '<|im_start|>system\n<|im_end|>\n<|im_start|>user\n\n## Role: 专业AI翻译助手\n- description: 你是一个专业的AI翻译助手，精通中文。\n\n## Skills\n1. 精通多种语言的翻译技巧\n2. 准确理解和传达原文含义\n3. 严格遵循输出格式要求\n4. 专业术语和品牌名称的处理能力\n\n## Rules\n1. 仅输出原文和翻译，不添加任何解释或评论。\n2. 严格按照指定的JSON格式提供翻译结果。\n3. 保持专业术语和品牌名称不变，除非有官方的中文译名。\n4. 确保翻译准确传达原文含义，同时保持中文的自然流畅。\n5. 如遇无法翻译的内容，保持原文不变，不做额外说明。\n6. 禁止在输出中包含任何额外的解释、注释或元数据。\n\n## Workflow\n1. 接收翻译任务：目标语言为中文，待翻译文本为"1 X 40 CONTAINERS CONTAINING 163 CARTONS OF VCAPS PLUS SIZE 0 NONPRINTED HPMC "。\n2. 分析文本，识别专业术语和品牌名称。\n3. 进行翻译，遵循翻译规则。\n4. 按照指定的JSON格式输出结果。\n\n## OutputFormat\n{\n "翻译后": ""\n}\n\n## Init\n你的任务是将给定的文本"1 X 40 CONTAINERS CONTAINING 163 CARTONS OF VCAPS PLUS SIZE 0 NONPRINTED HPMC "准确翻译成中文。请直接开始翻译工作，只返回要求的JSON格式结果。<|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.5, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=7840, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: [151644, 8948, 198, 151645, 198, 151644, 872, 271, 565, 15404, 25, 220, 99878, 15469, 105395, 110498, 198, 12, 4008, 25, 220, 56568, 101909, 104715, 15469, 105395, 110498, 3837, 114806, 104811, 3407, 565, 30240, 198, 16, 13, 10236, 110, 122, 31935, 101312, 102064, 9370, 105395, 102118, 198, 17, 13, 65727, 228, 33956, 115167, 107707, 103283, 109091, 198, 18, 13, 220, 100470, 106466, 66017, 68805, 101882, 198, 19, 13, 220, 99878, 116925, 33108, 100135, 29991, 9370, 54542, 99788, 271, 565, 22847, 198, 16, 13, 220, 99373, 66017, 103283, 33108, 105395, 3837, 16530, 42855, 99885, 104136, 57191, 85641, 8997, 17, 13, 220, 110439, 105146, 9370, 5370, 68805, 99553, 105395, 59151, 8997, 18, 13, 220, 100662, 99878, 116925, 33108, 100135, 29991, 105928, 3837, 106781, 18830, 100777, 9370, 104811, 102610, 13072, 8997, 19, 13, 10236, 94, 106, 32463, 105395, 102188, 107707, 103283, 109091, 3837, 91572, 100662, 104811, 9370, 99795, 110205, 8997, 20, 13, 69372, 99688, 101068, 105395, 104597, 3837, 100662, 103283, 105928, 3837, 109513, 108593, 66394, 8997, 21, 13, 10236, 99, 223, 81433, 18493, 66017, 15946, 102298, 99885, 108593, 9370, 104136, 5373, 25074, 68862, 57191, 23305, 20074, 3407, 565, 60173, 198, 16, 13, 46602, 98, 50009, 105395, 88802, 5122, 100160, 102064, 17714, 104811, 3837, 74193, 105395, 108704, 17714, 1, 16, 1599, 220, 19, 15, 16120, 6836, 4321, 16120, 6836, 1718, 220, 16, 21, 18, 78032, 29526, 3008, 647, 31400, 50, 58453, 25341, 220, 15, 20575, 24372, 1479, 472, 92523, 330, 8997, 17, 13, 58657, 97771, 108704, 3837, 102450, 99878, 116925, 33108, 100135, 29991, 8997, 18, 13, 32181, 249, 22243, 105395, 3837, 106466, 105395, 104190, 8997, 19, 13, 6567, 234, 231, 99331, 105146, 9370, 5370, 68805, 66017, 59151, 3407, 565, 9258, 4061, 198, 515, 220, 330, 105395, 33447, 788, 8389, 630, 565, 15690, 198, 103929, 88802, 20412, 44063, 89012, 22382, 9370, 108704, 1, 16, 1599, 220, 19, 15, 16120, 6836, 4321, 16120, 6836, 1718, 220, 16, 21, 18, 78032, 29526, 3008, 647, 31400, 50, 58453, 25341, 220, 15, 20575, 24372, 1479, 472, 92523, 330, 102188, 105395, 12857, 104811, 1773, 14880, 101041, 55286, 105395, 99257, 3837, 91680, 31526, 101882, 9370, 5370, 68805, 59151, 1773, 151645, 198, 151644, 77091, 198], lora_request: None, prompt_adapter_request: None. ERROR 10-16 11:05:16 client.py:244] TimeoutError('No heartbeat received from MQLLMEngine') ERROR 10-16 11:05:16 client.py:244] NoneType: None INFO: 61.171.72.231:58483 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error INFO: 61.171.72.231:32247 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error INFO: 61.171.72.231:25138 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error INFO: 61.171.72.231:10465 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error INFO: 61.171.72.231:10466 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error INFO: 61.171.72.231:32248 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error INFO: 61.171.72.231:28574 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error INFO: 61.171.72.231:21232 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error INFO: 61.171.72.231:17909 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error INFO: 61.171.72.231:40058 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/raid/demo/anaconda3/envs/vllm_latest/lib/python3.12/site-packages/starlette/responses.py", line 257, in call await wrap(partial(self.listen_for_disconnect, receive)) File "/raid/demo/anaconda3/envs/vllm_latest/lib/python3.12/site-packages/starlette/responses.py", line 253, in wrap await func() File "/raid/demo/anaconda3/envs/vllm_latest/lib/python3.12/site-packages/starlette/responses.py", line 230, in listen_for_disconnect message = await receive() ^^^^^^^^^^^^^^^ File "/raid/demo/anaconda3/envs/vllm_latest/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 555, in receive await self.message_event.wait() File "/raid/demo/anaconda3/envs/vllm_latest/lib/python3.12/asyncio/locks.py", line 212, in wait await fut asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f288c184c80
During handling of the above exception, another exception occurred:
During handling of the above exception, another exception occurred:
During handling of the above exception, another exception occurred:
During handling of the above exception, another exception occurred:
During handling of the above exception, another exception occurred:
hxhcreate
commented
4 weeks ago upgrade to v0.6.3 seems to solve this error
Clint-chan
commented
3 days ago
still
Your current environment
Model Input Dumps
No response
🐛 Describe the bug
(demo_vllm) demo@dgx03:/raid/xinference/modelscope/hub/qwen/Qwen2-72B-Instruct/logs$ tail -f vllm_20240927.log frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7f5cf7ba9897 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libc10.so) frame #1: c10d::ProcessGroupNCCL::WorkNCCL::checkTimeout(std::optional<std::chrono::duration<long, std::ratio<1l, 1000l> > >) + 0x1d2 (0x7f5cf8e82c62 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so) frame #2: c10d::ProcessGroupNCCL::watchdogHandler() + 0x1a0 (0x7f5cf8e87a80 in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so) frame #3: c10d::ProcessGroupNCCL::ncclCommWatchdog() + 0x10c (0x7f5cf8e88dcc in /raid/demo/anaconda3/envs/vllm/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so) frame #4: + 0xdbbf4 (0x7f5d44931bf4 in /raid/demo/anaconda3/envs/vllm/bin/../lib/libstdc++.so.6)
frame #5: + 0x8609 (0x7f5d4618b609 in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #6: clone + 0x43 (0x7f5d45f56353 in /lib/x86_64-linux-gnu/libc.so.6)
/raid/demo/anaconda3/envs/vllm/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '
Before submitting a new issue...