MaiziXiao
commented
1 month ago You might choose the wrong python interpreter to run start the job
Open wyzh0912 opened 1 month ago
MaiziXiao
commented
1 month ago You might choose the wrong python interpreter to run start the job
wyzh0912
commented
1 month ago I performed a test using the following code. import argparse import os import os.path as osp import random import sys import time from typing import Any import importlib.util
def is_package_installed(package_name): package_spec = importlib.util.find_spec(package_name) return package_spec is not None
测试 mmengine 包是否安装 package_name = "mmengine" if is_package_installed(package_name): print(f"{package_name} is installed.") else: print(f"{package_name} is not installed.")
from mmengine.config import Config, ConfigDict from mmengine.utils import mkdir_or_exist
The results are as following
/mnt/data131/junyi/code/opencompass$ CUDA_VISIBLE_DEVICES=2 python run.py configs/eval_llama3.py -w outputs/demo --debug
mmengine is installed.
07/17 10:49:43 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored.
07/17 10:49:43 - OpenCompass - DEBUG - Modules of opencompass's partitioner registry have been automatically imported from opencompass.partitioners
07/17 10:49:43 - OpenCompass - DEBUG - Get class SizePartitioner from "partitioner" registry in "opencompass"
07/17 10:49:43 - OpenCompass - DEBUG - An SizePartitioner instance is built from registry, and its implementation can be found in opencompass.partitioners.size
07/17 10:49:43 - OpenCompass - DEBUG - Key eval.runner.task.judge_cfg not found in config, ignored.
07/17 10:49:43 - OpenCompass - DEBUG - Key eval.runner.task.dump_details not found in config, ignored.
07/17 10:49:43 - OpenCompass - DEBUG - Key eval.given_pred not found in config, ignored.
07/17 10:49:43 - OpenCompass - DEBUG - Additional config: {}
07/17 10:49:43 - OpenCompass - INFO - Partitioned into 15 tasks.
07/17 10:49:43 - OpenCompass - DEBUG - Task 0: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_0]
07/17 10:49:43 - OpenCompass - DEBUG - Task 1: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_1]
07/17 10:49:43 - OpenCompass - DEBUG - Task 2: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_2]
07/17 10:49:43 - OpenCompass - DEBUG - Task 3: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_3]
07/17 10:49:43 - OpenCompass - DEBUG - Task 4: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_4]
07/17 10:49:43 - OpenCompass - DEBUG - Task 5: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_5]
07/17 10:49:43 - OpenCompass - DEBUG - Task 6: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_6]
07/17 10:49:43 - OpenCompass - DEBUG - Task 7: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_law_7]
07/17 10:49:43 - OpenCompass - DEBUG - Task 8: [llama-3-8b-instruct-hf/lukaemon_mmlu_moral_scenarios,llama-3-8b-instruct-hf/lukaemon_mmlu_miscellaneous]
07/17 10:49:43 - OpenCompass - DEBUG - Task 9: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_psychology,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_psychology,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_macroeconomics,llama-3-8b-instruct-hf/lukaemon_mmlu_elementary_mathematics]
07/17 10:49:43 - OpenCompass - DEBUG - Task 10: [llama-3-8b-instruct-hf/lukaemon_mmlu_moral_disputes,llama-3-8b-instruct-hf/lukaemon_mmlu_prehistory,llama-3-8b-instruct-hf/lukaemon_mmlu_philosophy,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_biology,llama-3-8b-instruct-hf/lukaemon_mmlu_nutrition,llama-3-8b-instruct-hf/lukaemon_mmlu_professional_accounting]
07/17 10:49:43 - OpenCompass - DEBUG - Task 11: [llama-3-8b-instruct-hf/lukaemon_mmlu_professional_medicine,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_mathematics,llama-3-8b-instruct-hf/lukaemon_mmlu_clinical_knowledge,llama-3-8b-instruct-hf/lukaemon_mmlu_security_studies,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_microeconomics,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_world_history,llama-3-8b-instruct-hf/lukaemon_mmlu_conceptual_physics,llama-3-8b-instruct-hf/lukaemon_mmlu_marketing]
07/17 10:49:43 - OpenCompass - DEBUG - Task 12: [llama-3-8b-instruct-hf/lukaemon_mmlu_human_aging,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_statistics,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_us_history,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_chemistry,llama-3-8b-instruct-hf/lukaemon_mmlu_sociology,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_geography,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_government_and_politics,llama-3-8b-instruct-hf/lukaemon_mmlu_college_medicine,llama-3-8b-instruct-hf/lukaemon_mmlu_world_religions,llama-3-8b-instruct-hf/lukaemon_mmlu_virology]
07/17 10:49:43 - OpenCompass - DEBUG - Task 13: [llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_european_history,llama-3-8b-instruct-hf/lukaemon_mmlu_logical_fallacies,llama-3-8b-instruct-hf/lukaemon_mmlu_astronomy,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_physics,llama-3-8b-instruct-hf/lukaemon_mmlu_electrical_engineering,llama-3-8b-instruct-hf/lukaemon_mmlu_college_biology,llama-3-8b-instruct-hf/lukaemon_mmlu_anatomy,llama-3-8b-instruct-hf/lukaemon_mmlu_human_sexuality,llama-3-8b-instruct-hf/lukaemon_mmlu_formal_logic,llama-3-8b-instruct-hf/lukaemon_mmlu_international_law,llama-3-8b-instruct-hf/lukaemon_mmlu_econometrics,llama-3-8b-instruct-hf/lukaemon_mmlu_machine_learning,llama-3-8b-instruct-hf/lukaemon_mmlu_public_relations,llama-3-8b-instruct-hf/lukaemon_mmlu_jurisprudence,llama-3-8b-instruct-hf/lukaemon_mmlu_management]
07/17 10:49:43 - OpenCompass - DEBUG - Task 14: [llama-3-8b-instruct-hf/lukaemon_mmlu_college_physics,llama-3-8b-instruct-hf/lukaemon_mmlu_college_chemistry,llama-3-8b-instruct-hf/lukaemon_mmlu_college_computer_science,llama-3-8b-instruct-hf/lukaemon_mmlu_college_mathematics,llama-3-8b-instruct-hf/lukaemon_mmlu_abstract_algebra,llama-3-8b-instruct-hf/lukaemon_mmlu_global_facts,llama-3-8b-instruct-hf/lukaemon_mmlu_computer_security,llama-3-8b-instruct-hf/lukaemon_mmlu_medical_genetics,llama-3-8b-instruct-hf/lukaemon_mmlu_high_school_computer_science,llama-3-8b-instruct-hf/lukaemon_mmlu_business_ethics,llama-3-8b-instruct-hf/lukaemon_mmlu_us_foreign_policy]
07/17 10:49:43 - OpenCompass - DEBUG - Modules of opencompass's runner registry have been automatically imported from opencompass.runners
07/17 10:49:43 - OpenCompass - DEBUG - Get class LocalRunner from "runner" registry in "opencompass"
07/17 10:49:43 - OpenCompass - DEBUG - An LocalRunner instance is built from registry, and its implementation can be found in opencompass.runners.local
07/17 10:49:43 - OpenCompass - DEBUG - Modules of opencompass's task registry have been automatically imported from opencompass.tasks
07/17 10:49:43 - OpenCompass - DEBUG - Get class OpenICLInferTask from "task" registry in "opencompass"
07/17 10:49:43 - OpenCompass - DEBUG - An OpenICLInferTask instance is built from registry, and its implementation can be found in opencompass.tasks.openicl_infer
mmengine is not installed.
Traceback (most recent call last):
File "/mnt/data131/junyi/code/opencompass/opencompass/tasks/openicl_infer.py", line 21, in
For the first time, the mmengine package in installed, but for the second time , the mmengine package is not installed
acylam
commented
1 month ago @wyzh0912 Were you using a virtual environment? Can you perhaps check the environment used to run the code? Something like:
which python
pip list | grep mmengine
Prerequisite
Type
I'm evaluating with the officially supported tasks/models/datasets.
Environment
{'CUDA available': True, 'CUDA_HOME': '/usr/local/cuda-12.4', 'GCC': 'gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0', 'GPU 0,1,2,3,4,5,6,7': 'NVIDIA H800 PCIe', 'MMEngine': '0.10.4', 'MUSA available': False, 'NVCC': 'Cuda compilation tools, release 12.4, V12.4.131', 'OpenCV': '4.9.0', 'PyTorch': '2.3.0', 'PyTorch compiling details': 'PyTorch built with:\n' ' - GCC 9.3\n' ' - C++ Version: 201703\n' ' - Intel(R) oneAPI Math Kernel Library Version ' '2023.1-Product Build 20230303 for Intel(R) 64 ' 'architecture applications\n' ' - Intel(R) MKL-DNN v3.3.6 (Git Hash ' '86e6af5974177e513fd3fee58425e1063e7f1361)\n' ' - OpenMP 201511 (a.k.a. OpenMP 4.5)\n' ' - LAPACK is enabled (usually provided by ' 'MKL)\n' ' - NNPACK is enabled\n' ' - CPU capability usage: AVX512\n' ' - CUDA Runtime 12.1\n' ' - NVCC architecture flags: ' '-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90\n' ' - CuDNN 8.9.2\n' ' - Magma 2.6.1\n' ' - Build settings: BLAS_INFO=mkl, ' 'BUILD_TYPE=Release, CUDA_VERSION=12.1, ' 'CUDNN_VERSION=8.9.2, ' 'CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, ' 'CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 ' '-fabi-version=11 -fvisibility-inlines-hidden ' '-DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO ' '-DLIBKINETO_NOROCTRACER -DUSE_FBGEMM ' '-DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK ' '-DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE ' '-O2 -fPIC -Wall -Wextra -Werror=return-type ' '-Werror=non-virtual-dtor -Werror=bool-operation ' '-Wnarrowing -Wno-missing-field-initializers ' '-Wno-type-limits -Wno-array-bounds ' '-Wno-unknown-pragmas -Wno-unused-parameter ' '-Wno-unused-function -Wno-unused-result ' '-Wno-strict-overflow -Wno-strict-aliasing ' '-Wno-stringop-overflow -Wsuggest-override ' '-Wno-psabi -Wno-error=pedantic ' '-Wno-error=old-style-cast -Wno-missing-braces ' '-fdiagnostics-color=always -faligned-new ' '-Wno-unused-but-set-variable ' '-Wno-maybe-uninitialized -fno-math-errno ' '-fno-trapping-math -Werror=format ' '-Wno-stringop-overflow, LAPACK_INFO=mkl, ' 'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, ' 'PERF_WITH_AVX512=1, TORCH_VERSION=2.3.0, ' 'USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=1, ' 'USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, ' 'USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, ' 'USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, ' 'USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, ' 'USE_ROCM_KERNEL_ASSERT=OFF, \n', 'Python': '3.10.14 (main, Mar 21 2024, 16:24:04) [GCC 11.2.0]', 'TorchVision': '0.18.0+cu121', 'numpy_random_seed': 2147483648, 'opencompass': '0.2.4+cce5b6f', 'sys.platform': 'linux'}
Reproduces the problem - code/configuration sample
openical_eval.py
import argparse import copy import fnmatch import math import os import os.path as osp import re import statistics import sys import time from collections import Counter from inspect import signature from typing import List, Optional
from mmengine.config import Config, ConfigDict from mmengine.utils import mkdir_or_exist
Reproduces the problem - command or script
CUDA_VISIBLE_DEVICES=2 python run.py configs/eval_llama3.py -w outputs/demo --debug
Reproduces the problem - error message
Traceback (most recent call last): File "/mnt/data131/junyi/code/opencompass/opencompass/tasks/openicl_infer.py", line 9, in
from mmengine.config import Config, ConfigDict
ModuleNotFoundError: No module named 'mmengine'
Other information
The
mmenginepackage is already in the environment; however, the error "No module named 'mmengine'" still occurs when runningrun.py.