1737686924
commented
5 days ago 我也遇到了这个问题，下载了 https://huggingface.co/BUAADreamer这个下面的模型 报错：The input provided to the model are wrong. The number of image tokens is 0 while the number of image given to the model is 1. This prevents correct indexing and breaks batch generation.
脚本如下：
ASCEND_RT_VISIBLE_DEVICES=5 python src/train.py \
--visual_inputs true \
--stage sft \
--do_train \
--include_num_input_tokens_seen true \
--model_name_or_path /data/applications/lmd-formal/backend/BaseModels/Yi-VL-6B-hf \
--dataset mllm_demo \
--template yi_vl \
--finetuning_type lora \
--lora_target all\
--output_dir saves/Yi-VL-6B-hf/lora/sft \
--overwrite_cache \
--overwrite_output_dir \
--cutoff_len 1024 \
--preprocessing_num_workers 16 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 8 \
--lr_scheduler_type cosine \
--logging_steps 1 \
--save_steps 100 \
--eval_steps 100 \
--evaluation_strategy steps \
--load_best_model_at_end \
--learning_rate 1e-4 \
--num_train_epochs 3.0 \
--val_size 0.1 \
--ddp_timeout 180000000 \
--plot_loss \
--warmup_ratio 0.1 \
--bf16
BUAADreamer
我想在LLama-Factory上推理Yi-VL-6B，下载01-ai/Yi-VL-6B上的模型，模型加载报错，查看报错信息需要下载 Download LLaVA-1.5 models from: https://huggingface.co/llava-hf Download Yi-VL models from: https://huggingface.co/BUAADreamer 想问一下能都支持01-ai/Yi-VL-6B的模型。