When position embedding, why do you not use angle but only xywh?

chagmgang commented 4 months ago

I think that when sine positional embedding the rotation box of xywha, the angle should also be reflected in the positional embedding, but this was not done in the code below. Can you tell why?
https://github.com/httle/ARS-DETR/blob/0d96c5e792dcca34793570c7acd67bb1068561c1/mmrotate/models/utils/dn_ars_rotated_transformer.py#L472

wokaikaixinxin commented 1 month ago

We also note the issue. Angles, position, and size are all necessary to represent oriented objects. Current transformer-based methods encode only position and size using vanilla PE, but neglect angles.
To encode the angle of oriented boxes in addition to position and size, we propose Gaussian PE in our paper (TGRS 2024) OrientedFormer: An End-to-End Transformer-Based Oriented Object Detector in Remote Sensing Images. Our paper link: https://ieeexplore.ieee.org/document/10669376 Our codes link: https://github.com/wokaikaixinxin/OrientedFormer Our codes support the newest mmrotate-1.x Welcome to cite OrientedFormer.

@ARTICLE{10669376,
  author={Zhao, Jiaqi and Ding, Zeyu and Zhou, Yong and Zhu, Hancheng and Du, Wen-Liang and Yao, Rui and El Saddik, Abdulmotaleb},
  journal={IEEE Transactions on Geoscience and Remote Sensing}, 
  title={OrientedFormer: An End-to-End Transformer-Based Oriented Object Detector in Remote Sensing Images}, 
  year={2024},
  volume={62},
  number={},
  pages={1-16},
  keywords={Encoding;Object detection;Proposals;Detectors;Remote sensing;Current transformers;Position measurement;End-to-end detectors;oriented object detection;positional encoding (PE);remote sensing;transformer},
  doi={10.1109/TGRS.2024.3456240}}

chagmgang commented 1 month ago

@wokaikaixinxin thank you.

httle / ARS-DETR

When position embedding, why do you not use angle but only xywh? #18