OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.
03/28 05:19:05 - OpenCompass - INFO - Task [LightllmAPI/lukaemon_mmlu_high_school_european_history]
03/28 05:19:08 - OpenCompass - INFO - Start inferencing [LightllmAPI/lukaemon_mmlu_high_school_european_history]
0%| | 0/165 [00:00<?, ?it/s]
100%|██████████| 165/165 [00:00<00:00, 1855389.17it/s]
[2024-03-28 05:19:09,235] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
0%| | 0/2 [00:00<?, ?it/s]--- Logging error ---
Traceback (most recent call last):
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/site-packages/requests/models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/adk/opencompass/opencompass/models/lightllm_api.py", line 76, in _generate
response = raw_response.json()
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/site-packages/requests/models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/logging/__init__.py", line 1100, in emit
msg = self.format(record)
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/logging/__init__.py", line 943, in format
return fmt.format(record)
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/site-packages/mmengine/logging/logger.py", line 124, in format
result = logging.Formatter.format(self, record)
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/logging/__init__.py", line 678, in format
record.message = record.getMessage()
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/logging/__init__.py", line 368, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/threading.py", line 973, in _bootstrap
self._bootstrap_inner()
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/concurrent/futures/thread.py", line 83, in _worker
work_item.run()
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/adk/opencompass/opencompass/models/lightllm_api.py", line 83, in _generate
self.logger.error('JsonDecode error, got',
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/logging/__init__.py", line 1506, in error
self._log(ERROR, msg, args, **kwargs)
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/logging/__init__.py", line 1624, in _log
self.handle(record)
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/logging/__init__.py", line 1634, in handle
self.callHandlers(record)
File "/home/adk/miniconda3/envs/opencompass/lib/python3.10/site-packages/mmengine/logging/logger.py", line 306, in callHandlers
handler.handle(record)
Message: 'JsonDecode error, got'
Arguments: ("b'Internal Server Error'",)
先决条件
问题类型
我正在使用官方支持的任务/模型/数据集进行评估。
环境
重现问题 - 代码/配置示例
重现问题 - 命令或脚本
python run.py configs/eval_lightllm.py
重现问题 - 错误信息
In terminal:
In the log file:
其他信息
No response