likyoo / SegEarth-OV

SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images
48 stars 0 forks source link

Questions about experimental results reproduction and dataset preparation: #8

Open yeelinsen opened 9 hours ago

yeelinsen commented 9 hours ago

I tried to reproduce the experimental results in the paper, but I encountered some problems.

  1. Regarding the dataset preparation, I found it difficult to prepare the dataset for building extraction
    a. WHU Building Dataset (Aerial&Sat II), here it is difficult to download the corresponding dataset accurately. b. The Inria dataset does not seem to provide a GT for the test set after downloading, and the test set includes only 27 images if we follow the data split of Link.

Considering the convenience of reproduction, could you please provide the download link of the dataset you used to reproduce the experimental results in the article?

  1. I prepared the dataset exactly according to the guideline in “dataset_prepare.md”, and used the default configs file to conduct experiments on several datasets, including openearthmap, loveda, iSAID, potsdam, vainghen, uavid, udd5, chn6-cug, deepglobe_road, massachusetts_road, spacenet_road, wbs-si. The experimental results are consistent with those in the paper, except for OpenEarthMap. The test result on the OpenEarthMap dataset was: 39.75 (40.3 in the paper).

image

likyoo commented 6 hours ago

Thank you for your meticulous reproduction! 1.a. The WHU_Aerial data can be downloaded from here, and the Sat II can be downloaded from here. 1.b. You can download the training data of Inria and run this script to get testing data. I am sorry that we seem to have made some errors in the description of this dataset in the appendix.

  1. Regarding the OpenEarthMap dataset, as we state in the paper, xBD data is excluded. But recently I realized that the xBD data doesn't seem to have been completely removed and the correct mIoU is about 40.6%.

Thank you again for your interest in our work! If there are any further queries, don't hesitate to contact us.