search

X-PLUG / mPLUG-Owl

mPLUG-Owl: The Powerful Multi-modal Large Language Model Family
https://www.modelscope.cn/studios/damo/mPLUG-Owl
MIT License
2.33k stars 176 forks source link

The performance is different from the results shown in the paper. #73

Closed gray311 closed 1 year ago

gray311 commented 1 year ago

I deployed mplug-owl locally and tried to input several pictures given in the paper, but the results were not very ideal. May I ask what is the reason? If it is affected by the parameters of generate, can you tell me a set of suitable parameters? (temperature, top_k,...)

gray311 commented 1 year ago

For example 1：

What is unusual about this picture, please describe it in detail. The unusual aspect of this scene is a man standing on top of a laundry basket, which is being pulled by a yellow taxi. The taxi is moving down a busy street and the man appears to be balancing on top of the basket as it moves along the pavement. It is an unusual and potentially hazardous sight to see a man standing on a moving laundry basket, especially in a busy city setting. 1

MAGAer13 commented 1 year ago

Since we have retrained the model with the latest version. There might be some difference in the results. Besides, the prompt you provided is different what we tried in the paper.

And the top_k sampling might lead to the difference results, but the overall performance is similar.

gray311 commented 1 year ago

Thanks for your reply @MAGAer13