search

BIT-DA / SePiCo

[TPAMI 2023 ESI Highly Cited Paper] SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic Segmentation https://arxiv.org/abs/2204.08808
https://arxiv.org/abs/2204.08808
Other
112 stars 7 forks source link

Question about running logit of the uda #18

Closed QChhh123 closed 1 year ago

QChhh123 commented 1 year ago

Thank you for your great work, I'm really interested in it.

Due to my poor coding ability, there are some questions about the framework. In your config file, we can know that you define uda part as a segmentor, https://github.com/BIT-DA/SePiCo/blob/0774b56e2c566e315a25997b915dd68076606149/mmseg/models/uda/uda_decorator.py#L30 but in the model part there is a segmentor too. https://github.com/BIT-DA/SePiCo/blob/0774b56e2c566e315a25997b915dd68076606149/configs/_base_/models/daformer_sepaspp_proj_mitb5.py#L11 Why there are two segmentors, and what's the running logit of that. Looking forward to your reply.

KiwiXR commented 1 year ago

Hi @QChhh123 , thanks for your interest in our work!

Guess you have noticed that this design is borrowed from DAFormer, therefore more official explanation could be acquired there :)

I highly appreciate this design, as it cleverly seperates method interface from model interface in an OOP manner.

In short, all segmentors inherited from BaseSegmentor(i.e., those in mmseg/models/segmentors) only define operations on the model itself, while those inherited from UDADecorator(i.e., those in mmseg/models/uda) actually define the details of the methods.

For the running logic behind it, I would suggest debugging yourself for details, but here are some hints to save your time:

  1. Take a look at experiments.py, the config comes from multiple files, including https://github.com/BIT-DA/SePiCo/blob/0774b56e2c566e315a25997b915dd68076606149/experiments.py#L72-L73 https://github.com/BIT-DA/SePiCo/blob/0774b56e2c566e315a25997b915dd68076606149/experiments.py#L17-L23 and https://github.com/BIT-DA/SePiCo/blob/0774b56e2c566e315a25997b915dd68076606149/experiments.py#L87 As you can see, taking DAFormer-based SePiCo as an example, the method config is sepico.py, while the model config is daformer_sepaspp_proj_mitb5.py.

  2. When we build the train model, we are actually building the method instance: https://github.com/BIT-DA/SePiCo/blob/0774b56e2c566e315a25997b915dd68076606149/tools/train.py#L143-L145 then in build_train_model(), https://github.com/BIT-DA/SePiCo/blob/0774b56e2c566e315a25997b915dd68076606149/mmseg/models/builder.py#L51-L56 then it's more of a mmcv thing, and the result is that a method instance of class cfg.uda['type'](in this case SePiCo) is built from config cfg.uda, including the default_args.

  3. Note that during initialization of method instance SePiCo, the model is built inside it: https://github.com/BIT-DA/SePiCo/blob/0774b56e2c566e315a25997b915dd68076606149/mmseg/models/uda/uda_decorator.py#L35 Therefore, we actually define the method(instance of SePiCo inherited from UDADecorator), which contains the daformer_sepaspp_proj_mitb5 model(instance of EncoderDecoderProjector inherited from BaseSegmentor).

  4. To conclude, we have a method instance of SePiCo(inherited from UDADecorator), and a model instance of EncoderDecoderProjector(inherited from BaseSegmentor). The model instance is a member variable of the method instance. In a method, if we would like to operate on the model, we do like this: https://github.com/BIT-DA/SePiCo/blob/0774b56e2c566e315a25997b915dd68076606149/mmseg/models/uda/sepico.py#L326-L327

The cool thing is that we only care about the model training and testing in BaseSegmentor and its derived classes, and only care about method design (eased by instance methods of model class) in UDADecorator and its derived classes.

Hope this does help, and please feel free to ask if I fail to make it clear.

QChhh123 commented 1 year ago

Thank you very much for your detailed answer, which made my mind suddenly clear.