yaosenJ / CoalQA

使用煤矿历史事故案例,事故处理报告、安全规程规章制度、技术文档、煤矿从业人员入职考试题库等数据,微调internlm2模型实现针对煤矿事故和煤矿安全知识的智能问答。
35 stars 7 forks source link

请教一个问题 #6

Open renlingjie opened 1 week ago

renlingjie commented 1 week ago

文档挺详细的,按照文档一步一步来,但是在进行微调的时候有如下报错。因为我是在Windows11上进行的,不知道这个问题您能否给出一些意见?

(internlm2) D:\pythondev\PyProjects\XTuner>xtuner train D:\pythondev\PyProjects\xtuner-0.1.18\config\internlm2_1_8b_qlora_alpaca_e3_copy.py --work-dir  D:\pythondev\PyProjects\xtuner-0.1.18\trained
[2024-10-17 11:08:31,009] [INFO] [real_accelerator.py:219:get_accelerator] Setting ds_accelerator to cuda (auto detect)
10/17 11:08:31 - mmengine - WARNING - WARNING: command error: '[WinError 2] 系统找不到指定的文件。'!
10/17 11:08:31 - mmengine - WARNING -
    Arguments received: ['xtuner', 'train', 'D:\\pythondev\\PyProjects\\xtuner-0.1.18\\config\\internlm2_1_8b_qlora_alpaca_e3_copy.py', '--work-dir', 'D:\\pythondev\\PyProjects\\xtuner-0.1.18\\trained']. xtuner commands use the following syntax:

        xtuner MODE MODE_ARGS ARGS

        Where   MODE (required) is one of ('list-cfg', 'copy-cfg', 'log-dataset', 'check-custom-dataset', 'train', 'test', 'chat', 'convert', 'preprocess', 'mmbench', 'eval_refcoco')
                MODE_ARG (optional) is the argument for specific mode
                ARGS (optional) are the arguments for specific command

    Some usages for xtuner commands: (See more by using -h for specific command!)

        1. List all predefined configs:
            xtuner list-cfg
        2. Copy a predefined config to a given path:
            xtuner copy-cfg $CONFIG $SAVE_FILE
        3-1. Fine-tune LLMs by a single GPU:
            xtuner train $CONFIG
renlingjie commented 1 week ago

路径应该是没有问题的,佐证的例子:我想去和预处理模型对话,也会报同样的错误: image

(internlm2) D:\pythondev\PyProjects\XTuner>xtuner chat D:\pythondev\PyProjects\xtuner-0.1.18\model\internlm2-chat-7b
[2024-10-17 11:12:50,559] [INFO] [real_accelerator.py:219:get_accelerator] Setting ds_accelerator to cuda (auto detect)
10/17 11:12:50 - mmengine - WARNING - WARNING: command error: '[WinError 2] 系统找不到指定的文件。'!
10/17 11:12:50 - mmengine - WARNING -
    Arguments received: ['xtuner', 'chat', 'D:\\pythondev\\PyProjects\\xtuner-0.1.18\\model\\internlm2-chat-7b']. xtuner commands use the following syntax:

        xtuner MODE MODE_ARGS ARGS
yaosenJ commented 5 days ago
  1. 可能xtuner没有安装好,xtuner --help 来检查是否可用。
  2. xtuner train "D:\pythondev\PyProjects\xtuner-0.1.18\config\internlm2_1_8b_qlora_alpaca_e3_copy.py" --work-dir "D:\pythondev\PyProjects\xtuner-0.1.18\trained"
renlingjie commented 4 days ago
  1. 可能xtuner没有安装好,xtuner --help 来检查是否可用。
  2. xtuner train "D:\pythondev\PyProjects\xtuner-0.1.18\config\internlm2_1_8b_qlora_alpaca_e3_copy.py" --work-dir "D:\pythondev\PyProjects\xtuner-0.1.18\trained"

感谢回复,不过xtuner是没什么问题的,包括配置文件都是通过xtuner命令下载的,至于您提供的第二条思路我之前也试过了,引号、斜杠、进到目录使用相对路径,以及使用dir命令确认路径对应的文件属性等。。我在A100的Ubuntu系统上跑就没问题,windows上就有这样的问题,挺奇怪。。