Is there a method to get the indexes of nouns in the original sentence within a specific scene graph?

zhuang-li / FactualSceneGraph

FACTUAL benchmark dataset, the pre-trained textual scene graph parser trained on FACTUAL.

https://arxiv.org/pdf/2305.17497.pdf

95 stars 12 forks source link

Is there a method to get the indexes of nouns in the original sentence within a specific scene graph? #4

Closed tiesanguaixia closed 6 months ago

tiesanguaixia commented 6 months ago

Thanks a lot for the favorable work and code release! I want to figure out if there is a method to get the indexes of nouns in the original sentence within a specific scene graph? For example, a input sentence is "a tall man is playing basketball and a fat man is playing football.", the output scene graph will be "(man, play, basketball), (man, play, football)". How can I know which person in the original sentence does these two words "man" refer to? Thank you in advance!

zhuang-li commented 6 months ago

I provided parsers that generate the indexes of the entities, which are "lizhuang144/flan-t5-small/base/large-VG-factual-sg-id". You can find them in the readme. Ideally, the parsers should output " (man, play, basketball), (man:1, play, football) ". However, I have not tested them on this specific case. Based on my observations, the number of instances with indexes is limited in the FACTUAL dataset. Therefore, it might depend on the model's ability to learn such cases.