search

zhuang-li / FactualSceneGraph

FACTUAL benchmark dataset, the pre-trained textual scene graph parser trained on FACTUAL.
https://arxiv.org/pdf/2305.17497.pdf
92 stars 12 forks source link

Can the pretrained model be used directly on images? #6

Closed atharvadeore999 closed 1 month ago

atharvadeore999 commented 1 month ago

Can we use the pretrained model on images to generate scene graph .If yes please let me know the steps required to perform it.

zhuang-li commented 1 month ago

No, the pretrained models are only for parsing texts. To parse images, you need to train the model yourself. There are image annotations in this dataset while I didn't perform related experiments.

atharvadeore999 commented 1 month ago

Can you please help me by suggesting some good models that will help me generate scene graphs from images? Your help will be greatly appreciated

zhuang-li commented 1 month ago

Sorry, I did some literature survey two years ago. By that time, none of the image scene graph parsers can work. They just generated a lot of noise. And this is also why I included the image annotation but I didn’t proceed. Because I feel the text parser at least can have useful downstream applications while I couldn’t know what the application would be with a poorly performed image scene graph parser.

However, that was before the era of LLMs, not sure whether the newest multimodal LLM can solve the problem. Or whether the image scene graph parsing problem is still worth solving?

atharvadeore999 @.***>于2024年6月27日 周四上午12:44写道：

Can you please help me by suggesting some good models that will help me generate scene graphs from images? Your help will be greatly appreciated

— Reply to this email directly, view it on GitHub https://github.com/zhuang-li/FactualSceneGraph/issues/6#issuecomment-2191891889, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACTSNJ36LGXXN2YDIFIW5T3ZJLHTPAVCNFSM6AAAAABJ4BO6SOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJRHA4TCOBYHE . You are receiving this because you commented.Message ID: @.***>

atharvadeore999 commented 1 month ago

Thank you so much for letting me know

On Thu, Jun 27, 2024 at 7:40 PM Zhuang Li @.***> wrote:

Closed #6 https://github.com/zhuang-li/FactualSceneGraph/issues/6 as completed.

— Reply to this email directly, view it on GitHub https://github.com/zhuang-li/FactualSceneGraph/issues/6#event-13318183894, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATV6GZ3PDDN47JYOZSUSM2LZJQMMDAVCNFSM6AAAAABJ4BO6SOVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJTGMYTQMJYGM4DSNA . You are receiving this because you authored the thread.Message ID: @.***>