search

X-PLUG / mPLUG-Owl

mPLUG-Owl: The Powerful Multi-modal Large Language Model Family
https://www.modelscope.cn/studios/damo/mPLUG-Owl
MIT License
2.25k stars 171 forks source link

The implementation is error in forward method #107

Closed zipzou closed 1 year ago

zipzou commented 1 year ago

See line at: https://github.com/X-PLUG/mPLUG-Owl/blob/main/mplug_owl/modeling_mplug_owl.py#L1448C9-L1448C25

In the forward method of the MplugOwlForConditionalGeneration model, there are three mask arguments are optional: non_padding_mask, non_media_mask, prompt_mask. These masks will be used to get loss mask, but your code has no additional process while them passed with None. And your generate method has been rewritten without invoking forward, thus this error would not happened.

image

Like this, if masks were None, this method raises None Exception directly.

MAGAer13 commented 1 year ago

During training, these masks would not be None. See the part of dataset.