Source Only Training

[BjoernMichele]

Dear @DZhaoXd ,

Thank you very much for your work and making the code available.

I am particularly interested in the code that you used for training the source only model in the GTA5-to-Cityscapes setting (without domain generalization techniques). I tried my luck with the source only code provided by DSU. Is that also the code you used ? Or could you point me to the code you used ?

While in 2 runs with the DSU code, I obtained source-only checkpoints that were in the range of your source-only performance (one was 37.9 %mIoU and one was even 39.5 %mIoU). However, when using DT-ST on them, both had a maximum performance of about 46% mIoU on cityscapes. However, with the checkpoint that you provide I reach with DT-ST the about 52% mIoU on cityscapes.

Is there some ingredient on the source-only training that I am missing, or some adaptation/normalization of the training data ?

Thank you very much again, With best regards, Bjoern

[DZhaoXd]

Thanks for your attention. Could you please provide the log files you trained using different pre-training models?

[DZhaoXd]

As you said, we used DSU's pre-trained model, and when I reproduced DSU, I obtained a similar performance to what was reported in the DSU paper. Additionally, I discovered that DSU may have a small bug in the provided code, involving perturbations at the feature layer that don't seem to load on deeper layers. I fixed this bug when reproducing the DSU, making it easier to reproduce the performance of the DSU. Maybe this problem is bothering you.

[BjoernMichele]

Thank you very much for your prompt response. Please find attached the two logs of the training runs. log_for_37_9.txt 2nd_run_log.txt

Just to make sure, for DSU you trained also for 20K iterations ?

I did not change the DSU code, if you point me to the perturbations I am happy to also change that.

Thank you very much :)

[DZhaoXd]

From the log, it seems that the performance of some main categories in the pre-trained model you provided is too poor, such as road and sidewalk. The poor initial performance of road and sidewalk will introduce a lot of noise into self-training. Even if DTST is used, this problem does not seem to be well improved.

I think this phenomenon is understandable under the source-free setting, because source-free is a setting with very scarce resources, which makes the model unable to be supported from other aspects (for example, like UDA, it can use other domain tag data). Therefore, once the pre-trained model is too poor, it seems that the existing source-free methods are somewhat helpless.

Regarding the changes to DSU, since it has been a long time, I just looked through the previous code but could not find the change record. I have been busy preparing for ECCV recently. Maybe I will have more energy to help you solve the problem after the submission is over.

Can you try the pre-trained model training I provide first? Or can I help you somewhere else?

[DZhaoXd]

By the way, was your pre-trained model trained using the original code of DSU, or the code under DTST?

[BjoernMichele]

Dear @DZhaoXd,

Thanks again for such a quick response, and good luck for the ECCV submission.

For the training of the source-only model I used entirely the code of DSU.

When I train from the checkpoint provided by you, I obtain a score of about 52 %mIoU (like what you reported, which makes me confident that the DT-ST part works fine).

Thank you very much again for your fast investigation, if you at some point come accross the change record, I am happy if you let it me know.

Best,

[DZhaoXd]

Thanks!
I just uploaded a training source-only code. I think you can use the following script to train source-only in the DTST code environment. It may help you run a better source-only model.

python train_src.py -cfg configs/deeplabv2_ur101_src.yaml OUTPUT_DIR results/src/

Please take care to import the correct pre-trained model in that file of line 276~279. resnet.py.

Good luck!

[BjoernMichele]

Thank you very much, that is very kind :) I will give it a try and let you know :)

[BjoernMichele]

Dear @DZhaoXd,

Thank you very much again for your prompt response and providing your source-only training script. With this source-only training it works fine for me, the DT-ST adaptation provides the same performances as with the checkpoint that you provide.

With best regards, Bjoern