Can UniMatch be used for Domain Adaptation?

[mr-intj-22]

Hi Young, great work!

I was wondering if this method can perform well for the Domain Adaptation task. I have a dataset (street driving domain) that is unlabelled and would like to include it in the unlabelled dataset. While the labeled dataset would be something like cityscapes, and maybe, other labeled datasets like (KITTI, GTA, BDD100K, etc).

Any suggestions? would it work?

[LiheYoung]

Hi, of course. I have also made some attempts on UniMatch for domain adaptation. It can work.

One key modification is that you should CutMix labeled (source domain) and unlabeled (target domain) images in domain adaptation, rather than mixing among unlabeled images in semi-supervised learning.

[mr-intj-22]

I see that's great! do you have any results from the Domain adaptation experiment?

I understand, so should the (source domain) image be also strongly perturbed? or the weakly perturbed one is enough to CutMix?

another point is, should the (source domain) image be CutMix-ed into the (target domain) image or the opposite? I assume the (target domain) image pixels should dominate the image but I would like some insights if possible.

[LiheYoung]

Sorry that I cannot remember the results clearly.

But I remember the much better choice is to paste a rectangle region of the source-domain labeled image onto the target-domain image. The target-domain region should be larger and dominate the learning because our final test domain is the target domain. The source-domain image is CutMixed onto the target-domain image just to narrow the domain gap.

And the weakly perturbed source-domain image is enough for CutMix, because it uses human labels as the supervision. To some extent, human supervision itself can serve as a perturbation for our model. Also, strong perturbations will also possibly damage its original source-domain property.

[mr-intj-22]

Thank you a lot! I will sure let you know how it goes!

[LiheYoung]

I am going to close this issue now. If you obtain the results, you can re-open it and share them here! Thank you. :)

[tanveer6715]

Thank you a lot! I will sure let you know how it goes!

Hi, Have you try domain adoption and obtained the results. If so then please share your experience how it works. Thanks

[mr-intj-22]

@tanveer6715 Unfortunately, my hardware didn't allow me to train. I managed to get mIoU=0.72 on cityscapes using the 1/16 split with a smaller image size of 640 and a batch of 2 images only to accumulate the hardware limitations. We'll try to get better hardware and try.

[tanveer6715]

@tanveer6715 Unfortunately, my hardware didn't allow me to train. I managed to get mIoU=0.72 on cityscapes using the 1/16 split with a smaller image size of 640 and a batch of 2 images only to accumulate the hardware limitations. We'll try to get better hardware and try.

@mr-intj-22 ok thanks. But would you share what kind of changes you made for domain adaptation using UniMatch?