在自定义数据集上微调多模态大模型

[SunLang115]

请问：在cli上运行时，如何使用自定义的数据集； 例如：微调MiniCPM-V时候，找不到对应的sft.sh，只找到了minicpm_2b_sft_chat和minicpm_moe_8x2b；然而利用cli： # Experimental environment: A10, 3090, V100, ... # 10GB GPU memory CUDA_VISIBLE_DEVICES=0 swift sft \ --model_type minicpm-v-3b-chat \ --dataset coco-mini-en-2 \

是否需要修改--dataset？
还是怎么做，我找不到相关方法，能否提供更加易懂和全面的信息及方法。

[hjh0119]

请问：在cli上运行时，如何使用自定义的数据集

https://github.com/modelscope/swift/blob/main/docs/source/LLM/%E8%87%AA%E5%AE%9A%E4%B9%89%E4%B8%8E%E6%8B%93%E5%B1%95.md#%E8%87%AA%E5%AE%9A%E4%B9%89%E6%95%B0%E6%8D%AE%E9%9B%86

[SunLang115]

请问：在cli上运行时，如何使用自定义的数据集

https://github.com/modelscope/swift/blob/main/docs/source/LLM/%E8%87%AA%E5%AE%9A%E4%B9%89%E4%B8%8E%E6%8B%93%E5%B1%95.md#%E8%87%AA%E5%AE%9A%E4%B9%89%E6%95%B0%E6%8D%AE%E9%9B%86

抱歉没有理解这个文档的内容，找不到与MiniCPM-V模型相对应的sfh.sh 能否提示

[hjh0119]

找不到与MiniCPM-V模型相对应的sfh.sh

https://github.com/modelscope/swift/blob/main/docs/source/Multi-Modal/minicpm-v%E6%9C%80%E4%BD%B3%E5%AE%9E%E8%B7%B5.md 自定义数据集使用--custom_train_dataset_path参数

[SunLang115]

找不到与MiniCPM-V模型相对应的sfh.sh

https://github.com/modelscope/swift/blob/main/docs/source/Multi-Modal/minicpm-v%E6%9C%80%E4%BD%B3%E5%AE%9E%E8%B7%B5.md 自定义数据集使用--custom_train_dataset_path参数

命令行这样： (swift) [sunkaijie@localhost swift]$ CUDA_VISIBLE_DEVICES=0 swift sft \

--model_type minicpm-v-3b-chat \

--custom_train_dataset_path /home/sunkaijie/project/VisualGLM-6B/skin_data/skin_data_split/minicpm_data_train.json

出现这样的错误： [INFO:swift] PeftModelForCausalLM: 3439.3895M Params (4.4237M Trainable [0.1286%]), 10.4870M Buffers. [INFO:swift] Setting model.config.use_cache: False [INFO:swift] Downloading the dataset from ModelScope, datasetid: Traceback (most recent call last): File "/home/sunkaijie/project/swift/swift/cli/sft.py", line 5, in sft_main() File "/home/sunkaijie/project/swift/swift/utils/run_utils.py", line 31, in x_main result = llm_x(args, **kwargs) File "/home/sunkaijie/project/swift/swift/llm/sft.py", line 134, in llm_sft train_dataset, val_dataset = get_dataset( File "/home/sunkaijie/project/swift/swift/llm/utils/dataset.py", line 1658, in get_dataset dataset = get_function( File "/home/sunkaijie/project/swift/swift/llm/utils/dataset.py", line 1728, in get_custom_dataset train_dataset = load_dataset_from_local(train_subset_split_list, File "/home/sunkaijie/project/swift/swift/llm/utils/dataset.py", line 1713, in load_dataset_from_local df = transform_jsonl_to_df(obj_list) File "/home/sunkaijie/project/swift/swift/utils/np_utils.py", line 15, in transform_jsonl_to_df for k, v in obj.items(): AttributeError: 'str' object has no attribute 'items'

[SunLang115]

找不到与MiniCPM-V模型相对应的sfh.sh

https://github.com/modelscope/swift/blob/main/docs/source/Multi-Modal/minicpm-v%E6%9C%80%E4%BD%B3%E5%AE%9E%E8%B7%B5.md 自定义数据集使用--custom_train_dataset_path参数

是在命令行直接加上：“ --custom_train_dataset_path （后面跟地址）”吗？

[SunLang115]

找不到与MiniCPM-V模型相对应的sfh.sh

https://github.com/modelscope/swift/blob/main/docs/source/Multi-Modal/minicpm-v%E6%9C%80%E4%BD%B3%E5%AE%9E%E8%B7%B5.md 自定义数据集使用--custom_train_dataset_path参数

找不到MiniCPM-V模型的sfh.sh，如何使用自定义数据集微调

[Jintao-Huang]

https://github.com/modelscope/swift/blob/main/docs/source/Multi-Modal/minicpm-v-2%E6%9C%80%E4%BD%B3%E5%AE%9E%E8%B7%B5.md

这里有写sh的