MaureenZOU
commented
1 year ago Thanks so much for asking this question and the interest in our work! It is indeed a quite important question and we properly handle this in our experiments. For a single image we add the gt from different dataset together, and for the image that have overlap with evaluated dataset we remove them, please refer to the image here in X-Decoder ReadMe we statistic the overlap and properly handle them. That's why we didn't evaluate on all the ref segmentation dataset but only choose one as the exclusion image will be too much!
I'd like to express my appreciation for your excellent work. it is both engaging and insightful. However, I have some confusion regarding the experiment detailed in Chapter 4.
In the paper, you mentioned using a combination of Ref-COCO, Ref-COCOg, and Ref-COCO+ for COCO image annotations in the referring segmentation task. Then, you report your evaluation on Ref-COCOg. While I find this approach interesting, I'm not quite sure what you mean by "combination." Additionally, I am concerned about the potential for data leakage since Ref-COCO, Ref-COCOg, and Ref-COCO+ are three types of annotations on the same image dataset, which might lead to overlap between the training and test sets of different annotations. Could you please provide further clarification on this experimental part? Thank you!