salesforce / LAVIS

LAVIS - A One-stop Library for Language-Vision Intelligence
BSD 3-Clause "New" or "Revised" License
9.6k stars 942 forks source link

How to avoid an intellectual property in text output when using BLIP2? #582

Open starush opened 9 months ago

starush commented 9 months ago

I am using the Salesforce/blip2-opt-2.7b model (on RTX 3070 8Gb) with blip2 for images captioning.

Sometimes, the generated text includes irrelevant or unwarranted intellectual property, such as 'Pineapple wallpaper iphone 6' in response to a simple pineapple image. It also occasionally includes names of TV shows even when they are not related. Moreover, when the image is a generic smartphone, it often generates 'iPhone.'

How can I avoid this issue? The presence of intellectual property in the generated text is highly undesirable.

shams2023 commented 9 months ago

我使用 Salesforce/blip2-opt-2.7b 模型(在 RTX 3070 8Gb 上)和 blip2 进行图像字幕。

有时,生成的文本包含不相关或无根据的知识产权,例如响应简单的菠萝图像的“菠萝壁纸 iphone 6”。它偶尔也会包含电视节目的名称,即使它们不相关。此外,当图像是通用智能手机时,它通常会生成“iPhone”。

我怎样才能避免这个问题?生成的文本中存在知识产权是非常不受欢迎的。

Hi, brother, have you solved your problem? I also encountered the same problem. For things that are not in my image, he also generates text descriptions, which is very annoying for me. May I ask how to solve him? Or can we communicate to solve it? Wishing you a thriving academic career