search

lhoyer / HRDA

[ECCV22] Official Implementation of HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation
Other
231 stars 32 forks source link

Impressive work, but still some issues #3

Closed BoltenWang-Meta closed 2 years ago

BoltenWang-Meta commented 2 years ago

Hi, Dr. Hoyer. Thanks for your contribution to the community. This is indeed a nice work, which inspired me a lot. After reading the paper, may i summary the core idea is to combine the multi-resolutions to adapt context and fine-grained features. However, did you ever try to directly train on HR inputs via DAFormer? Moreover, the select of HR regions are random. So have you ever considered to select them according to sth. For there are certain feature distribution correlated with spatial.

lhoyer commented 2 years ago

Thank you very much for your interest in our work!

Yes, we also studied the behavior of DAFormer trained with HR inputs. The results can be seen in Fig. 4 for different crop sizes of HR inputs (orange curve). Further, we directly compare HRDA with DAFormer on HR inputs in Tab. 5. The crop size in this table is 768x768 pixels of HR inputs for DAFormer (denotes as HR_0.75) to have a comparable memory consumption as HRDA. This specific crop size for HR inputs is also commonly used in supervised learning. For further discussion of these results, please have a look at Sec. 5.4 and Sec. 5.5 Paragraph "Comparison with Naive HR".

We also considered selecting the HR crop regions based on sample-specific features. However, in these exploratory studies, we couldn't observe significant performance improvements.

BoltenWang-Meta commented 2 years ago

Thanks for your reply! It is useful to help me further understand the paper. But I still deem in generating pseudo prediction, randomly select HR region may introduce unstable prediction. Considering the training phase, the HR regions come from randomly cropping. In generating pseudo GT, the HR region comes from sliding window to cover the coarse image. Despite that HR sampler obeys a uniform distribution which is consistency with fully sliding sampler, the real situtaion is not like that when assigning a discrete probability distribution to a single image in training. This obviouly does not satisfy large numbers theorem. So I am wondering whether to directly learn to select a HR in training and in pseudo generation to place a restriction on the region distribution to keep consistancy.

BoltenWang-Meta commented 2 years ago

can make further promotion or stablize the pseudo GT generating.

lhoyer commented 2 years ago

I'm sorry but I don't get your point when you argue that the HR crop and the pseudo label regions are inconsistent and would cause an unstable pseudo label generation. Could you please explain this argument?

BoltenWang-Meta commented 2 years ago

I am sorry, this is not the place to make further discussion. I will contact you through email in days.