search

open-compass / opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.
https://opencompass.org.cn/
Apache License 2.0
4.17k stars 445 forks source link

[Bug] ModuleNotFoundError: No module named 'mmengine' #1328

Open wyzh0912 opened 4 months ago

wyzh0912 commented 4 months ago

Prerequisite

Type

I'm evaluating with the officially supported tasks/models/datasets.

Environment

{'CUDA available': True, 'CUDA_HOME': '/usr/local/cuda-12.4', 'GCC': 'gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0', 'GPU 0,1,2,3,4,5,6,7': 'NVIDIA H800 PCIe', 'MMEngine': '0.10.4', 'MUSA available': False, 'NVCC': 'Cuda compilation tools, release 12.4, V12.4.131', 'OpenCV': '4.9.0', 'PyTorch': '2.3.0', 'PyTorch compiling details': 'PyTorch built with:\n' ' - GCC 9.3\n' ' - C++ Version: 201703\n' ' - Intel(R) oneAPI Math Kernel Library Version ' '2023.1-Product Build 20230303 for Intel(R) 64 ' 'architecture applications\n' ' - Intel(R) MKL-DNN v3.3.6 (Git Hash ' '86e6af5974177e513fd3fee58425e1063e7f1361)\n' ' - OpenMP 201511 (a.k.a. OpenMP 4.5)\n' ' - LAPACK is enabled (usually provided by ' 'MKL)\n' ' - NNPACK is enabled\n' ' - CPU capability usage: AVX512\n' ' - CUDA Runtime 12.1\n' ' - NVCC architecture flags: ' '-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90\n' ' - CuDNN 8.9.2\n' ' - Magma 2.6.1\n' ' - Build settings: BLAS_INFO=mkl, ' 'BUILD_TYPE=Release, CUDA_VERSION=12.1, ' 'CUDNN_VERSION=8.9.2, ' 'CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, ' 'CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 ' '-fabi-version=11 -fvisibility-inlines-hidden ' '-DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO ' '-DLIBKINETO_NOROCTRACER -DUSE_FBGEMM ' '-DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK ' '-DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE ' '-O2 -fPIC -Wall -Wextra -Werror=return-type ' '-Werror=non-virtual-dtor -Werror=bool-operation ' '-Wnarrowing -Wno-missing-field-initializers ' '-Wno-type-limits -Wno-array-bounds ' '-Wno-unknown-pragmas -Wno-unused-parameter ' '-Wno-unused-function -Wno-unused-result ' '-Wno-strict-overflow -Wno-strict-aliasing ' '-Wno-stringop-overflow -Wsuggest-override ' '-Wno-psabi -Wno-error=pedantic ' '-Wno-error=old-style-cast -Wno-missing-braces ' '-fdiagnostics-color=always -faligned-new ' '-Wno-unused-but-set-variable ' '-Wno-maybe-uninitialized -fno-math-errno ' '-fno-trapping-math -Werror=format ' '-Wno-stringop-overflow, LAPACK_INFO=mkl, ' 'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, ' 'PERF_WITH_AVX512=1, TORCH_VERSION=2.3.0, ' 'USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=1, ' 'USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, ' 'USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, ' 'USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, ' 'USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, ' 'USE_ROCM_KERNEL_ASSERT=OFF, \n', 'Python': '3.10.14 (main, Mar 21 2024, 16:24:04) [GCC 11.2.0]', 'TorchVision': '0.18.0+cu121', 'numpy_random_seed': 2147483648, 'opencompass': '0.2.4+cce5b6f', 'sys.platform': 'linux'}

Reproduces the problem - code/configuration sample

openical_eval.py

import argparse import copy import fnmatch import math import os import os.path as osp import re import statistics import sys import time from collections import Counter from inspect import signature from typing import List, Optional

from mmengine.config import Config, ConfigDict from mmengine.utils import mkdir_or_exist

Reproduces the problem - command or script

CUDA_VISIBLE_DEVICES=2 python run.py configs/eval_llama3.py -w outputs/demo --debug

Reproduces the problem - error message

Traceback (most recent call last): File "/mnt/data131/junyi/code/opencompass/opencompass/tasks/openicl_infer.py", line 9, in from mmengine.config import Config, ConfigDict ModuleNotFoundError: No module named 'mmengine'

Other information

The mmengine package is already in the environment; however, the error "No module named 'mmengine'" still occurs when running run.py.

MaiziXiao commented 4 months ago

You might choose the wrong python interpreter to run start the job

wyzh0912 commented 4 months ago

I performed a test using the following code. import argparse import os import os.path as osp import random import sys import time from typing import Any import importlib.util

def is_package_installed(package_name): package_spec = importlib.util.find_spec(package_name) return package_spec is not None

测试 mmengine 包是否安装 package_name = "mmengine" if is_package_installed(package_name): print(f"{package_name} is installed.") else: print(f"{package_name} is not installed.")

from mmengine.config import Config, ConfigDict from mmengine.utils import mkdir_or_exist

The results are as following

/mnt/data131/junyi/code/opencompass$ CUDA_VISIBLE_DEVICES=2 python run.py configs/eval_llama3.py -w outputs/demo --debug mmengine is installed. 07/17 10:49:43 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored. 07/17 10:49:43 - OpenCompass - DEBUG - Modules of opencompass's partitioner registry have been automatically imported from opencompass.partitioners 07/17 10:49:43 - OpenCompass - DEBUG - Get class SizePartitioner from "partitioner" registry in "opencompass" 07/17 10:49:43 - OpenCompass - DEBUG - An SizePartitioner instance is built from registry, and its implementation can be found in opencompass.partitioners.size 07/17 10:49:43 - OpenCompass - DEBUG - Key eval.runner.task.judge_cfg not found in config, ignored. 07/17 10:49:43 - OpenCompass - DEBUG - Key eval.runner.task.dump_details not found in config, ignored. 07/17 10:49:43 - OpenCompass - DEBUG - Key eval.given_pred not found in config, ignored. 07/17 10:49:43 - OpenCompass - DEBUG - Additional config: {} 07/17 10:49:43 - OpenCompass - INFO - Partitioned into 15 tasks. 07/17 10:49:43 - OpenCompass - DEBUG - Task 0: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_0] 07/17 10:49:43 - OpenCompass - DEBUG - Task 1: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_1] 07/17 10:49:43 - OpenCompass - DEBUG - Task 2: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_2] 07/17 10:49:43 - OpenCompass - DEBUG - Task 3: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_3] 07/17 10:49:43 - OpenCompass - DEBUG - Task 4: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_4] 07/17 10:49:43 - OpenCompass - DEBUG - Task 5: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_5] 07/17 10:49:43 - OpenCompass - DEBUG - Task 6: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_6] 07/17 10:49:43 - OpenCompass - DEBUG - Task 7: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_7] 07/17 10:49:43 - OpenCompass - DEBUG - Task 8: [llama-3-8b-instruct-hf/lukaemon_mmlu_moral_scenarios,llama-3-8b-instruct-hf/lukaemon_mmlu_miscellaneous] 07/17 10:49:43 - OpenCompass - DEBUG - Task 9: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_psychology,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_psychology,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_macroeconomics,llama-3-8b-instruct-hf/lukaemon_mmlu_elementary_mathematics] 07/17 10:49:43 - OpenCompass - DEBUG - Task 10: [llama-3-8b-instruct-hf/lukaemon_mmlu_moral_disputes,llama-3-8b-instruct-hf/lukaemon_mmlu_prehistory,llama-3-8b-instruct-hf/lukaemon_mmlu_philosophy,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_biology,llama-3-8b-instruct-hf/lukaemon_mmlu_nutrition,llama-3-8b-instruct-hf/lukaemon_mmlu_professional_accounting] 07/17 10:49:43 - OpenCompass - DEBUG - Task 11: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_medicine,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_mathematics,llama-3-8b-instruct-hf/lukaemon_mmlu_clinical_knowledge,llama-3-8b-instruct-hf/lukaemon_mmlu_security_studies,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_microeconomics,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_world_history,llama-3-8b-instruct-hf/lukaemon_mmlu_conceptual_physics,llama-3-8b-instruct-hf/lukaemon_mmlu_marketing] 07/17 10:49:43 - OpenCompass - DEBUG - Task 12: [llama-3-8b-instruct-hf/lukaemon_mmlu_human_aging,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_statistics,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_us_history,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_chemistry,llama-3-8b-instruct-hf/lukaemon_mmlu_sociology,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_geography,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_government_and_politics,llama-3-8b-instruct-hf/lukaemon_mmlu_college_medicine,llama-3-8b-instruct-hf/lukaemon_mmlu_world_religions,llama-3-8b-instruct-hf/lukaemon_mmlu_virology] 07/17 10:49:43 - OpenCompass - DEBUG - Task 13: [llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_european_history,llama-3-8b-instruct-hf/lukaemon_mmlu_logical_fallacies,llama-3-8b-instruct-hf/lukaemon_mmlu_astronomy,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_physics,llama-3-8b-instruct-hf/lukaemon_mmlu_electrical_engineering,llama-3-8b-instruct-hf/lukaemon_mmlu_college_biology,llama-3-8b-instruct-hf/lukaemon_mmlu_anatomy,llama-3-8b-instruct-hf/lukaemon_mmlu_human_sexuality,llama-3-8b-instruct-hf/lukaemon_mmlu_formal_logic,llama-3-8b-instruct-hf/lukaemon_mmlu_international_law,llama-3-8b-instruct-hf/lukaemon_mmlu_econometrics,llama-3-8b-instruct-hf/lukaemon_mmlu_machine_learning,llama-3-8b-instruct-hf/lukaemon_mmlu_public_relations,llama-3-8b-instruct-hf/lukaemon_mmlu_jurisprudence,llama-3-8b-instruct-hf/lukaemon_mmlu_management] 07/17 10:49:43 - OpenCompass - DEBUG - Task 14: [llama-3-8b-instruct-hf/lukaemon_mmlu_college_physics,llama-3-8b-instruct-hf/lukaemon_mmlu_college_chemistry,llama-3-8b-instruct-hf/lukaemon_mmlu_college_computer_science,llama-3-8b-instruct-hf/lukaemon_mmlu_college_mathematics,llama-3-8b-instruct-hf/lukaemon_mmlu_abstract_algebra,llama-3-8b-instruct-hf/lukaemon_mmlu_global_facts,llama-3-8b-instruct-hf/lukaemon_mmlu_computer_security,llama-3-8b-instruct-hf/lukaemon_mmlu_medical_genetics,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_computer_science,llama-3-8b-instruct-hf/lukaemon_mmlu_business_ethics,llama-3-8b-instruct-hf/lukaemon_mmlu_us_foreign_policy] 07/17 10:49:43 - OpenCompass - DEBUG - Modules of opencompass's runner registry have been automatically imported from opencompass.runners 07/17 10:49:43 - OpenCompass - DEBUG - Get class LocalRunner from "runner" registry in "opencompass" 07/17 10:49:43 - OpenCompass - DEBUG - An LocalRunner instance is built from registry, and its implementation can be found in opencompass.runners.local 07/17 10:49:43 - OpenCompass - DEBUG - Modules of opencompass's task registry have been automatically imported from opencompass.tasks 07/17 10:49:43 - OpenCompass - DEBUG - Get class OpenICLInferTask from "task" registry in "opencompass" 07/17 10:49:43 - OpenCompass - DEBUG - An OpenICLInferTask instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_infer mmengine is not installed. Traceback (most recent call last): File "/mnt/data131/junyi/code/opencompass/opencompass/tasks/openicl_infer.py", line 21, in from mmengine.config import Config, ConfigDict ModuleNotFoundError: No module named 'mmengine'

For the first time, the mmengine package in installed, but for the second time , the mmengine package is not installed

acylam commented 4 months ago

@wyzh0912 Were you using a virtual environment? Can you perhaps check the environment used to run the code? Something like:

which python
pip list | grep mmengine
Rosejacka commented 1 month ago

May I ask if you have solved it? I have also encountered a similar problem