search

zhu-xlab / UniDA

Code for TGRS paper "Universal Domain Adaptation for Remote Sensing Image Scene Classification"
35 stars 2 forks source link

Unable to reproduce source free domain adaptation results #3

Closed ujwal-k03 closed 7 months ago

ujwal-k03 commented 10 months ago

Hi, the MA with source data yields results close to what is claimed by the paper. However, the MA without source data yields extremely off and unstable results.

Here are the final accuracy scores for RSSCN->UCM source free MA:

[0.58, 0.09, 0.09, 0.59, 0.44, 0.3535714285714286]
    tensor([[0.3573]])

Some things that I have tried:

  1. Using source data instead of synthetic data in the SDG-MA/main.py file. This seems to alleviate the problem and the results are identical to the MA with source data.
  2. Double checking the loss functions and using the generator in eval mode.

I have attached an image that is the synthetic data generated. Please let me know if this is what is to be expected.

Lastly, here is my configuration:

python==3.11.5
cuda==12.2
torch==2.0.1

images-100

Any sort of help would be appreciated to rectify my reproduction, will appreciate any clarifications. Many thanks.

QingsongXu123 commented 9 months ago

Hi, Thanks for your attention and reproduction of our work! According to the results you have reproduced so far, the core problem is that there are some problems with the generated data and distributions. Firstly, to reproduce our results, please use the pre-trained model on RSSCN provided by us (link is here: https://drive.google.com/file/d/1EPX0qbq0kqanXDed4iy2kDhreguv-qJz/view?usp=drive_link). Secondly, please check your data generation part, the training is likely insufficient. If you have further questions about the training details, you can contact us by email: qingsong.xu@tum.de. Best, Qingsong

ujwal-k03 commented 7 months ago

Hi, Thanks for your attention and reproduction of our work! According to the results you have reproduced so far, the core problem is that there are some problems with the generated data and distributions. Firstly, to reproduce our results, please use the pre-trained model on RSSCN provided by us (link is here: https://drive.google.com/file/d/1EPX0qbq0kqanXDed4iy2kDhreguv-qJz/view?usp=drive_link). Secondly, please check your data generation part, the training is likely insufficient. If you have further questions about the training details, you can contact us by email: qingsong.xu@tum.de. Best, Qingsong

I have sent you a mail. Kindly reply at the earliest. Many thanks.