TypeError: can only concatenate list (not "int") to list 这个错误怎么解决

[zhly0]

Describe the bug What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程，最好有截图)

自定义数据集微调mninicpm-v-v2 命令： CUDA_VISIBLE_DEVICES=2 python /opt/conda/envs/xxxxx/lib/python3.10/site-packages/swift/cli/sft.py --model_type minicpm-v-v2 --dataset ./test.jsonl 报错： File "/opt/conda/envs/xxxxx/lib/python3.10/site-packages/swift/llm/utils/template.py", line 1198, in encode labels[:idx - 1] + [-100] len(placeholder_id) TypeError: can only concatenate list (not "int") to list 这个错误怎么解决 查看文件 /opt/conda/envs/xxxxx/lib/python3.10/site-packages/swift/llm/utils/template.py 对应代码： labels = ( labels[:idx - 1] + [-100] len(placeholder_id)

• labels[idx + 2]) labels为list而labels[idx + 2]为int，等于是list = list+int报错，貌似是要将后面的int变成list[int]

python版本3.10

不知道是我的错误还是什么其他导致的

[zhly0]

接下来使用以下sh script： swift sft \ --model_id_or_path OpenBMB/MiniCPM-V-2 \ --model_revision master \ --sft_type lora \ --tuner_backend peft \ --dtype bf16 \ --output_dir output \ --custom_train_dataset_path test.jsonl \ --train_dataset_sample -1 \ --num_train_epochs 1 \ --max_length 2048 \ --check_dataset_strategy warning \ --lora_rank 8 \ --lora_alpha 32 \ --lora_dropout_p 0.05 \ --lora_target_modules DEFAULT \ --gradient_checkpointing false \ --batch_size 1 \ --weight_decay 0.1 \ --learning_rate 1e-4 \ --gradient_accumulation_steps 16 \ --max_grad_norm 0.5 \ --warmup_ratio 0.03 \ --eval_steps 100 \ --save_steps 100 \ --save_total_limit 2 \ --logging_steps 10 \ --use_flash_attn false

报同样的错误。

[zhly0]

手动修正这行代码： 由 labels = ( labels[:idx - 1] + [-100] len(placeholder_id) + labels[idx + 2]) 变成 labels = ( labels[:idx - 1] + [-100] len(placeholder_id) + [labels[idx + 2]]) 该错误消失，但出现以下问题： return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing) ValueError: Expected input batch_size (1420) to match target batch_size (469).

改用官网的命令： CUDA_VISIBLE_DEVICES=2 swift sft --model_type minicpm-v-v2 --dataset coco-mini-en-2 错误也是一模一样

[zhly0]

pip list如下： absl-py 2.1.0 accelerate 0.24.0 addict 2.4.0 aiofiles 23.2.1 aiohttp 3.8.6 aiosignal 1.3.1 aliyun-python-sdk-core 2.15.1 aliyun-python-sdk-kms 2.16.2 altair 5.1.2 annotated-types 0.6.0 anyio 3.7.1 apex 0.1 appdirs 1.4.4 async-timeout 4.0.3 attrs 23.1.0 auto-gptq 0.6.0+cu118 bitsandbytes 0.41.0 blinker 1.7.0 cachetools 5.3.2 certifi 2023.7.22 cffi 1.16.0 charset-normalizer 3.3.1 chatglm-cpp 0.3.0 click 8.1.7 cmake 3.28.1 colorama 0.4.6 coloredlogs 15.0.1 contourpy 1.1.1 cpm-kernels 1.0.11 crcmod 1.7 cryptography 42.0.5 cycler 0.12.1 dacite 1.8.1 datasets 2.18.0 decorator 4.4.2 decord 0.6.0 deepspeed 0.10.0 diffusers 0.25.0 dill 0.3.7 distro 1.8.0 dnspython 2.4.2 docker-pycreds 0.4.0 docstring-parser 0.15 dropout-layer-norm 0.1 editdistance 0.8.1 einops 0.6.1 einops-exts 0.0.4 exceptiongroup 1.1.3 fastapi 0.109.0 ffmpy 0.3.1 filelock 3.12.4 fire 0.5.0 flash-attn 2.3.6 fonttools 4.43.1 frozenlist 1.4.0 fsspec 2023.10.0 fuzzywuzzy 0.18.0 gast 0.5.4 gekko 1.0.6 gitdb 4.0.11 GitPython 3.1.40 gradio 4.26.0 gradio_client 0.15.1 grpcio 1.62.1 h11 0.14.0 hjson 3.1.0 httpcore 1.0.5 httpx 0.27.0 huggingface-hub 0.20.2 humanfriendly 10.0 idna 3.4 imageio 2.34.0 imageio-ffmpeg 0.4.9 importlib-metadata 6.8.0 importlib_resources 6.4.0 jieba 0.42.1 Jinja2 3.1.2 jmespath 0.10.0 joblib 1.3.2 jsonlines 4.0.0 jsonschema 4.19.1 jsonschema-specifications 2023.7.1 kiwisolver 1.4.5 latex2mathml 3.76.0 linkify-it-py 2.0.2 lit 17.0.6 llmuses 0.3.0 lmdeploy 0.2.1 loguru 0.7.2 lxml 5.1.0 Markdown 3.5.1 markdown-it-py 2.2.0 markdown2 2.4.13 MarkupSafe 2.1.3 matplotlib 3.8.0 mdit-py-plugins 0.3.3 mdtex2html 1.2.0 mdurl 0.1.2 mmengine-lite 0.10.3 modelscope 1.13.3 moviepy 1.0.3 mpmath 1.3.0 ms-swift 2.0.0 multidict 6.0.4 multiprocess 0.70.15 networkx 3.2 ninja 1.11.1.1 nltk 3.8.1 numpy 1.26.1 nvidia-cublas-cu11 11.10.3.66 nvidia-cublas-cu12 12.1.3.1 nvidia-cuda-cupti-cu11 11.7.101 nvidia-cuda-cupti-cu12 12.1.105 nvidia-cuda-nvrtc-cu11 11.7.99 nvidia-cuda-nvrtc-cu12 12.1.105 nvidia-cuda-runtime-cu11 11.7.99 nvidia-cuda-runtime-cu12 12.1.105 nvidia-cudnn-cu11 8.5.0.96 nvidia-cudnn-cu12 8.9.2.26 nvidia-cufft-cu11 10.9.0.58 nvidia-cufft-cu12 11.0.2.54 nvidia-curand-cu11 10.2.10.91 nvidia-curand-cu12 10.3.2.106 nvidia-cusolver-cu11 11.4.0.1 nvidia-cusolver-cu12 11.4.5.107 nvidia-cusparse-cu11 11.7.4.91 nvidia-cusparse-cu12 12.1.0.106 nvidia-nccl-cu11 2.14.3 nvidia-nccl-cu12 2.18.1 nvidia-nvjitlink-cu12 12.3.52 nvidia-nvtx-cu11 11.7.91 nvidia-nvtx-cu12 12.1.105 openai 0.28.0 opencv-python 4.9.0.80 optimum 1.18.1 orjson 3.9.9 oss2 2.18.4 packaging 23.2 pandas 2.1.1 peft 0.10.0 Pillow 10.1.0 pip 23.3 platformdirs 4.1.0 plotly 5.20.0 ply 3.11 portalocker 2.8.2 proglog 0.1.10 protobuf 4.24.4 psutil 5.9.6 py-cpuinfo 9.0.0 pyarrow 13.0.0 pyarrow-hotfix 0.6 pycocoevalcap 1.2 pycocotools 2.0.7 pycparser 2.22 pycryptodome 3.20.0 pydantic 2.7.0 pydantic_core 2.18.1 pydeck 0.8.1b0 pydub 0.25.1 Pygments 2.16.1 pymongo 4.6.1 Pympler 1.0.1 pynvml 11.5.0 pyparsing 3.1.1 pytesseract 0.3.10 python-dateutil 2.8.2 python-docx 1.1.0 python-multipart 0.0.9 pytz 2023.3.post1 PyYAML 6.0.1 referencing 0.30.2 regex 2023.10.3 requests 2.31.0 requests-toolbelt 1.0.0 rich 13.6.0 rouge 1.0.1 rouge-chinese 1.0.3 rouge-score 0.1.2 rpds-py 0.10.6 ruff 0.3.7 sacrebleu 2.4.2 safetensors 0.4.0 scikit-learn 1.4.1.post1 scipy 1.11.3 seaborn 0.13.2 semantic-version 2.10.0 sentencepiece 0.1.99 sentry-sdk 1.45.0 setproctitle 1.3.3 setuptools 68.0.0 shellingham 1.5.4 shortuuid 1.0.11 shtab 1.6.4 simple-ddl-parser 1.0.4 simplejson 3.19.2 six 1.16.0 smmap 5.0.1 sniffio 1.3.0 sortedcontainers 2.4.0 sse-starlette 1.6.5 starlette 0.35.1 streamlit 1.28.2 svgwrite 1.4.3 sympy 1.12 tabulate 0.9.0 tenacity 8.2.3 tensorboard 2.16.2 tensorboard-data-server 0.7.2 tensorboardX 2.6.2.2 termcolor 2.3.0 threadpoolctl 3.3.0 tiktoken 0.5.1 timm 0.9.16 tokenizers 0.15.1 toml 0.10.2 tomli 2.0.1 tomlkit 0.12.0 toolz 0.12.0 torch 2.0.1 torchaudio 2.1.0+cu118 torchvision 0.15.2 tornado 6.3.3 tqdm 4.66.1 transformers 4.36.2 transformers-stream-generator 0.0.4 triton 2.0.0 trl 0.8.3 typer 0.12.3 typing_extensions 4.8.0 tyro 0.8.3 tzdata 2023.3 tzlocal 5.2 uc-micro-py 1.0.2 urllib3 2.0.7 uvicorn 0.23.2 validators 0.22.0 wandb 0.16.6 watchdog 3.0.0 wavedrom 2.0.3.post3 websockets 11.0.3 Werkzeug 3.0.2 wheel 0.41.2 xxhash 3.4.1 yacs 0.1.8 yapf 0.40.2 yarl 1.9.2 zipp 3.17.0

[zhly0]

ms-swift版本更新后，暂时未看见问题