search

Jingkang50 / OpenPSG

Benchmarking Panoptic Scene Graph Generation (PSG), ECCV'22
https://psgdataset.org
MIT License
421 stars 69 forks source link

How to get the features about objects and its relations #58

Closed TreezzZ closed 2 years ago

TreezzZ commented 2 years ago

Hi, Thanks for your great work!

I would like to use the features of objects and its relations for downstream tasks. Two-stage methods can get them from RoI head. Are one-stage methods cannot get them?

GSeanCDAT commented 2 years ago

Hi, thanks for your concern about our work! There are features of objects and relations in our one-stage models. For PSGFormer, the features of objects and relations correspond to the object queries and relation queries and they are associated with each other in the forward pass by computing the similarity. https://github.com/Jingkang50/OpenPSG/blob/23f9c89d43eeaa6e4ee57895d11ac9fb3546fd47/openpsg/models/relation_heads/psgformer_head.py#L277 For PSGTR, we model the obj-rel-obj triplet directly as triplet queries which indicates the feature of the relation and objects involved in the relation. https://github.com/Jingkang50/OpenPSG/blob/23f9c89d43eeaa6e4ee57895d11ac9fb3546fd47/openpsg/models/relation_heads/psgtr_head.py#L302 Unfortunately, we didn't output them directly in the forward pass. You may add them to the output of forward function for ypur interest.