执行llamafactory-cli api examples/inference/qwen2_vllm.yaml报错
2024-06-26 15:06:32.602948: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-06-26 15:06:32.738577: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable TF_ENABLE_ONEDNN_OPTS=0.
2024-06-26 15:06:32.776424: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-06-26 15:06:33.339303: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2024-06-26 15:06:33.339400: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2024-06-26 15:06:33.339411: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
[2024-06-26 15:06:36,114] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect)
/opt/conda/lib/python3.10/site-packages/pydantic/_internal/_config.py:334: UserWarning: Valid config keys have changed in V2:
'allow_population_by_field_name' has been renamed to 'populate_by_name'
'validate_all' has been renamed to 'validate_default'
warnings.warn(message, UserWarning)
/opt/conda/lib/python3.10/site-packages/pydantic/_internal/_fields.py:160: UserWarning: Field "model_persistencethreshold" has conflict with protected namespace "model".
You may be able to resolve this warning by setting model_config['protected_namespaces'] = ().
warnings.warn(
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/trl/import_utils.py", line 180, in _get_module
return importlib.import_module("." + module_name, self.name)
File "/opt/conda/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/opt/conda/lib/python3.10/site-packages/trl/trainer/dpo_trainer.py", line 46, in
from .utils import (
File "/opt/conda/lib/python3.10/site-packages/trl/trainer/utils.py", line 51, in
import deepspeed
File "/opt/conda/lib/python3.10/site-packages/deepspeed/init.py", line 22, in
from . import module_inject
File "/opt/conda/lib/python3.10/site-packages/deepspeed/module_inject/init.py", line 6, in
from .replace_module import replace_transformer_layer, revert_transformer_layer, ReplaceWithTensorSlicing, GroupQuantizer, generic_injection
File "/opt/conda/lib/python3.10/site-packages/deepspeed/module_inject/replace_module.py", line 778, in
from ..pipe import PipelineModule
File "/opt/conda/lib/python3.10/site-packages/deepspeed/pipe/init.py", line 6, in
from ..runtime.pipe import PipelineModule, LayerSpec, TiedLayerSpec
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/pipe/init.py", line 6, in
from .module import PipelineModule, LayerSpec, TiedLayerSpec
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/pipe/module.py", line 19, in
from ..activation_checkpointing import checkpointing
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/activation_checkpointing/checkpointing.py", line 25, in
from deepspeed.runtime.config import DeepSpeedConfig
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/config.py", line 29, in
from .zero.config import get_zero_config, ZeroStageEnum
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/zero/init.py", line 6, in
from .partition_parameters import ZeroParamType
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/zero/partition_parameters.py", line 681, in
class Init(InsertPostInitMethodToModuleSubClasses):
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/zero/partition_parameters.py", line 683, in Init
param_persistence_threshold = get_config_default(DeepSpeedZeroConfig, "param_persistence_threshold")
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/config_utils.py", line 116, in get_config_default
field_name).required, f"'{field_name}' is a required field and does not have a default value"
AttributeError: 'FieldInfo' object has no attribute 'required'. Did you mean: 'is_required'?
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/conda/bin/llamafactory-cli", line 5, in
from llamafactory.cli import main
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/init.py", line 3, in
from .cli import VERSION
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/cli.py", line 7, in
from . import launcher
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/launcher.py", line 1, in
from llamafactory.train.tuner import run_exp
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/train/tuner.py", line 11, in
from .dpo import run_dpo
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/train/dpo/init.py", line 1, in
from .workflow import run_dpo
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/train/dpo/workflow.py", line 11, in
from .trainer import CustomDPOTrainer
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/train/dpo/trainer.py", line 10, in
from trl import DPOTrainer
File "", line 1075, in _handle_fromlist
File "/opt/conda/lib/python3.10/site-packages/trl/import_utils.py", line 171, in getattr
value = getattr(module, name)
File "/opt/conda/lib/python3.10/site-packages/trl/import_utils.py", line 170, in getattr
module = self._get_module(self._class_to_module[name])
File "/opt/conda/lib/python3.10/site-packages/trl/import_utils.py", line 182, in _get_module
raise RuntimeError(
RuntimeError: Failed to import trl.trainer.dpo_trainer because of the following error (look up to see its traceback):
'FieldInfo' object has no attribute 'required'
qwen2_vllm.yaml文件内容如下 model_name_or_path: exports/qwen2-1.5b_lora_sft template: qwen infer_backend: vllm vllm_enforce_eager: true
执行llamafactory-cli api examples/inference/qwen2_vllm.yaml报错 2024-06-26 15:06:32.602948: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2024-06-26 15:06:32.738577: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable
TF_ENABLE_ONEDNN_OPTS=0. 2024-06-26 15:06:32.776424: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2024-06-26 15:06:33.339303: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory 2024-06-26 15:06:33.339400: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory 2024-06-26 15:06:33.339411: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. [2024-06-26 15:06:36,114] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect) /opt/conda/lib/python3.10/site-packages/pydantic/_internal/_config.py:334: UserWarning: Valid config keys have changed in V2:You may be able to resolve this warning by setting", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/opt/conda/lib/python3.10/site-packages/trl/trainer/dpo_trainer.py", line 46, in
from .utils import (
File "/opt/conda/lib/python3.10/site-packages/trl/trainer/utils.py", line 51, in
import deepspeed
File "/opt/conda/lib/python3.10/site-packages/deepspeed/init.py", line 22, in
from . import module_inject
File "/opt/conda/lib/python3.10/site-packages/deepspeed/module_inject/init.py", line 6, in
from .replace_module import replace_transformer_layer, revert_transformer_layer, ReplaceWithTensorSlicing, GroupQuantizer, generic_injection
File "/opt/conda/lib/python3.10/site-packages/deepspeed/module_inject/replace_module.py", line 778, in
from ..pipe import PipelineModule
File "/opt/conda/lib/python3.10/site-packages/deepspeed/pipe/init.py", line 6, in
from ..runtime.pipe import PipelineModule, LayerSpec, TiedLayerSpec
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/pipe/init.py", line 6, in
from .module import PipelineModule, LayerSpec, TiedLayerSpec
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/pipe/module.py", line 19, in
from ..activation_checkpointing import checkpointing
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/activation_checkpointing/checkpointing.py", line 25, in
from deepspeed.runtime.config import DeepSpeedConfig
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/config.py", line 29, in
from .zero.config import get_zero_config, ZeroStageEnum
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/zero/init.py", line 6, in
from .partition_parameters import ZeroParamType
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/zero/partition_parameters.py", line 681, in
class Init(InsertPostInitMethodToModuleSubClasses):
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/zero/partition_parameters.py", line 683, in Init
param_persistence_threshold = get_config_default(DeepSpeedZeroConfig, "param_persistence_threshold")
File "/opt/conda/lib/python3.10/site-packages/deepspeed/runtime/config_utils.py", line 116, in get_config_default
field_name).required, f"'{field_name}' is a required field and does not have a default value"
AttributeError: 'FieldInfo' object has no attribute 'required'. Did you mean: 'is_required'?
model_config['protected_namespaces'] = (). warnings.warn( Traceback (most recent call last): File "/opt/conda/lib/python3.10/site-packages/trl/import_utils.py", line 180, in _get_module return importlib.import_module("." + module_name, self.name) File "/opt/conda/lib/python3.10/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/opt/conda/bin/llamafactory-cli", line 5, in
from llamafactory.cli import main
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/init.py", line 3, in
from .cli import VERSION
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/cli.py", line 7, in
from . import launcher
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/launcher.py", line 1, in
from llamafactory.train.tuner import run_exp
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/train/tuner.py", line 11, in
from .dpo import run_dpo
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/train/dpo/init.py", line 1, in
from .workflow import run_dpo
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/train/dpo/workflow.py", line 11, in
from .trainer import CustomDPOTrainer
File "/data/jovyan/work/shuangye/LLaMA-Factory/src/llamafactory/train/dpo/trainer.py", line 10, in
from trl import DPOTrainer
File "", line 1075, in _handle_fromlist
File "/opt/conda/lib/python3.10/site-packages/trl/import_utils.py", line 171, in getattr
value = getattr(module, name)
File "/opt/conda/lib/python3.10/site-packages/trl/import_utils.py", line 170, in getattr
module = self._get_module(self._class_to_module[name])
File "/opt/conda/lib/python3.10/site-packages/trl/import_utils.py", line 182, in _get_module
raise RuntimeError(
RuntimeError: Failed to import trl.trainer.dpo_trainer because of the following error (look up to see its traceback):
'FieldInfo' object has no attribute 'required'