search

X-PLUG / mPLUG-Owl

mPLUG-Owl: The Powerful Multi-modal Large Language Model Family
https://www.modelscope.cn/studios/damo/mPLUG-Owl
MIT License
2.25k stars 171 forks source link

请问是否支持中文数据微调呀？May I finetune mPLUG-Owl with chinese image-text pair? #180

Open LianghuiGuo opened 10 months ago

LianghuiGuo commented 10 months ago

可否用中文的图文对进行SFT呢？ May I finetune mPLUG-Owl with chinese image-text pair?

MAGAer13 commented 10 months ago

Sure.

shaswati1 commented 10 months ago

@MAGAer13, while passing below one as input to MplugOwlForConditionalGeneration, it forces me to pass non_padding_mask, non_media_mask and prompt_mask as input too even though those are optional in the forward pass. Why is this the case?

inputs = {
    "pixel_values": image_pixel_values,
    "video_pixel_values": video_pixel_values,
    "input_ids": input_ids,
    "num_images": num_images_tensor,
    "num_videos": num_videos_tensor,
    "attention_mask": attention_mask,
}