search

open-compass / opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.
https://opencompass.org.cn/
Apache License 2.0
4.23k stars 451 forks source link

[Bug] #141

Closed wyzh0912 closed 1 year ago

wyzh0912 commented 1 year ago

描述该错误

运行评测脚本时，出现以下错误

Traceback (most recent call last): File "/d1/pub/wyzh/evaluation/opencompass/run.py", line 339, in main() File "/d1/pub/wyzh/evaluation/opencompass/run.py", line 210, in main tasks = partitioner(cfg) File "/d1/pub/wyzh/evaluation/opencompass/opencompass/partitioners/base.py", line 48, in call tasks = self.partition(models, datasets, work_dir, self.out_dir) File "/d1/pub/wyzh/evaluation/opencompass/opencompass/partitioners/size.py", line 67, in partition datasets = sorted(datasets, File "/d1/pub/wyzh/evaluation/opencompass/opencompass/partitioners/size.py", line 68, in key=lambda x: self.get_cost(x), File "/d1/pub/wyzh/evaluation/opencompass/opencompass/partitioners/size.py", line 188, in get_cost dataset = build_dataset_from_cfg(dataset) File "/d1/pub/wyzh/evaluation/opencompass/opencompass/utils/build.py", line 13, in build_dataset_from_cfg return LOAD_DATASET.build(dataset_cfg) File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/mmengine/registry/registry.py", line 570, in build return self.build_func(cfg, *args, *kwargs, registry=self) File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 98, in build_from_cfg obj_cls = registry.get(obj_type) File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/mmengine/registry/registry.py", line 451, in get self.import_from_location() File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/mmengine/registry/registry.py", line 376, in import_from_location import_module(loc) File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "/d1/pub/wyzh/evaluation/opencompass/opencompass/datasets/init.py", line 1, in from .afqmcd import # noqa: F401, F403 File "/d1/pub/wyzh/evaluation/opencompass/opencompass/datasets/afqmcd.py", line 7, in from .base import BaseDataset File "/d1/pub/wyzh/evaluation/opencompass/opencompass/datasets/base.py", line 6, in from opencompass.openicl import DatasetReader File "/d1/pub/wyzh/evaluation/opencompass/opencompass/openicl/init.py", line 2, in from .icl_evaluator import # noqa File "/d1/pub/wyzh/evaluation/opencompass/opencompass/openicl/icl_evaluator/init.py", line 4, in from .icl_hf_evaluator import # noqa File "/d1/pub/wyzh/evaluation/opencompass/opencompass/openicl/icl_evaluator/icl_hf_evaluator.py", line 4, in import evaluate File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/evaluate/init.py", line 29, in from .evaluation_suite import EvaluationSuite File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/evaluate/evaluation_suite/init.py", line 10, in from ..evaluator import evaluator File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/evaluate/evaluator/init.py", line 17, in from transformers.pipelines import SUPPORTED_TASKS as SUPPORTED_PIPELINE_TASKS File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/transformers/pipelines/init.py", line 44, in from .audio_classification import AudioClassificationPipeline File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/transformers/pipelines/audio_classification.py", line 21, in from .base import PIPELINE_INIT_ARGS, Pipeline File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/transformers/pipelines/base.py", line 36, in from ..modelcard import ModelCard File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/transformers/modelcard.py", line 48, in from .training_args import ParallelMode File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/transformers/training_args.py", line 67, in from accelerate import PartialState File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/accelerate/init.py", line 3, in from .accelerator import Accelerator File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/accelerate/accelerator.py", line 33, in from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/accelerate/checkpointing.py", line 24, in from .utils import ( File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/accelerate/utils/init.py", line 109, in from .launch import ( File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/accelerate/utils/launch.py", line 23, in from ..commands.config.config_args import SageMakerConfig File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/accelerate/commands/config/init.py", line 19, in from .config import config_command_parser File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/accelerate/commands/config/config.py", line 25, in from .sagemaker import get_sagemaker_input File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/accelerate/commands/config/sagemaker.py", line 35, in import boto3 # noqa: F401 File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/boto3/init.py", line 17, in from boto3.session import Session File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/boto3/session.py", line 17, in import botocore.session File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/botocore/session.py", line 26, in import botocore.client File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/botocore/client.py", line 15, in from botocore import waiter, xform_name File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/botocore/waiter.py", line 18, in from botocore.docs.docstring import WaiterDocstring File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/botocore/docs/init.py", line 15, in from botocore.docs.service import ServiceDocumenter File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/botocore/docs/service.py", line 14, in from botocore.docs.client import ClientDocumenter, ClientExceptionsDocumenter File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/botocore/docs/client.py", line 17, in from botocore.docs.example import ResponseExampleDocumenter File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/botocore/docs/example.py", line 13, in from botocore.docs.shape import ShapeDocumenter File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/botocore/docs/shape.py", line 19, in from botocore.utils import is_json_value_header File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/botocore/utils.py", line 37, in import botocore.httpsession File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/botocore/httpsession.py", line 45, in from urllib3.contrib.pyopenssl import ( File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/urllib3/contrib/pyopenssl.py", line 50, in import OpenSSL.crypto File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/OpenSSL/init.py", line 8, in from OpenSSL import crypto, SSL File "/home/llm/anaconda3/envs/safe-rlhf/lib/python3.10/site-packages/OpenSSL/crypto.py", line 3268, in _lib.OpenSSL_add_all_algorithms() AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'

环境信息

{'CUDA available': True, 'CUDA_HOME': '/home/llm/anaconda3/envs/safe-rlhf', 'GCC': 'gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0', 'GPU 0,1,2,3,4,5,6,7': 'NVIDIA A100-SXM4-40GB', 'MMEngine': '0.8.3', 'NVCC': 'Cuda compilation tools, release 11.7, V11.7.99', 'OpenCV': '4.8.0', 'PyTorch': '2.0.1', 'PyTorch compiling details': 'PyTorch built with:\n' ' - GCC 9.3\n' ' - C++ Version: 201703\n' ' - Intel(R) oneAPI Math Kernel Library Version ' '2023.1-Product Build 20230303 for Intel(R) 64 ' 'architecture applications\n' ' - Intel(R) MKL-DNN v2.7.3 (Git Hash ' '6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)\n' ' - OpenMP 201511 (a.k.a. OpenMP 4.5)\n' ' - LAPACK is enabled (usually provided by ' 'MKL)\n' ' - NNPACK is enabled\n' ' - CPU capability usage: AVX2\n' ' - CUDA Runtime 11.7\n' ' - NVCC architecture flags: ' '-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37\n' ' - CuDNN 8.5\n' ' - Magma 2.6.1\n' ' - Build settings: BLAS_INFO=mkl, ' 'BUILD_TYPE=Release, CUDA_VERSION=11.7, ' 'CUDNN_VERSION=8.5.0, ' 'CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, ' 'CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 ' '-fabi-version=11 -Wno-deprecated ' '-fvisibility-inlines-hidden -DUSE_PTHREADPOOL ' '-DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER ' '-DUSE_FBGEMM -DUSE_QNNPACK ' '-DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK ' '-DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC ' '-Wall -Wextra -Werror=return-type ' '-Werror=non-virtual-dtor -Werror=bool-operation ' '-Wnarrowing -Wno-missing-field-initializers ' '-Wno-type-limits -Wno-array-bounds ' '-Wno-unknown-pragmas -Wunused-local-typedefs ' '-Wno-unused-parameter -Wno-unused-function ' '-Wno-unused-result -Wno-strict-overflow ' '-Wno-strict-aliasing ' '-Wno-error=deprecated-declarations ' '-Wno-stringop-overflow -Wno-psabi ' '-Wno-error=pedantic -Wno-error=redundant-decls ' '-Wno-error=old-style-cast ' '-fdiagnostics-color=always -faligned-new ' '-Wno-unused-but-set-variable ' '-Wno-maybe-uninitialized -fno-math-errno ' '-fno-trapping-math -Werror=format ' '-Werror=cast-function-type ' '-Wno-stringop-overflow, LAPACK_INFO=mkl, ' 'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, ' 'PERF_WITH_AVX512=1, ' 'TORCH_DISABLE_GPU_ASSERTS=ON, ' 'TORCH_VERSION=2.0.1, USE_CUDA=ON, USE_CUDNN=ON, ' 'USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, ' 'USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, ' 'USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, ' 'USE_OPENMP=ON, USE_ROCM=OFF, \n', 'Python': '3.10.11 | packaged by conda-forge | (main, May 10 2023, 18:58:44) ' '[GCC 11.3.0]', 'TorchVision': '0.15.2+cu117', 'numpy_random_seed': 2147483648, 'opencompass': '0.1.0+e9b7b8a', 'sys.platform': 'linux'}

其他信息

No response

vansin commented 1 year ago

怎么运行的测评脚本呢？

wyzh0912 commented 1 year ago

python run.py configs/Self.py -w outputs/self 就像这样，self.py就是跟示例中一样的脚本，包括数据集的定义和模型的定义

gaotongxiao commented 1 year ago

Try this out:

pip install -U pyopenssl cryptography

Ref: https://stackoverflow.com/questions/74981558/error-updating-python3-pip-attributeerror-module-lib-has-no-attribute-openss