search

ZhenZHAO / iMAS

[CVPR'23] Instance-specific and Model-adaptive Supervision for Semi-supervised Semantic Segmentation
https://arxiv.org/abs/2211.11335
36 stars 5 forks source link

Question about the CutMix experiment #16

Closed Hugo-cell111 closed 1 year ago

Hugo-cell111 commented 1 year ago

Hi! I see you conduct the ablation experiments of Intensity-based augmentations and CutMix-based augmentations. But I wonder if there still exists obvious improvement between vanilla CutMix strategy and designed CutMix-based augmentation strategy, including your another work AugSeg accepted by CVPR2023? Since when I conduct my own experiment, I find that just applying vanilla CutMix can yield a satisfying performance. Thanks!

ZhenZHAO commented 1 year ago

Hi, pls check my log, I found an improvement from 76.1 to 76.9 in terms of the revised CutMix. Can you provide your log as well as your cutmix implementation? BTW, prefer doing exps on more labels with hundres of labels.

Hugo-cell111 commented 1 year ago

The log is here: output.log, and my CutMix implementation is the same as UniMatch's. There is no difference. BTW which log shows the improvement from 76.1 to 76.9, I can't find it. Thanks!

ZhenZHAO commented 1 year ago

I see. I only show the figures in CVPR's submission. Pls check https://openreview.net/pdf?id=AtyO3IZYVEy, I give the detailed performance in Tab. 4.

If you are using Lihe's implementation of Cutmix in Unimatch, you will indeed get slightly better performance. based on my understanding: 1) better randomness of the cut region (you can wrote some test code to examine it) 2) two batches for mixup (better differences). But in my opinion, you can still get higher performance using my revised one. What is the setting of your provided log?? I cannot see any details?

ZhenZHAO commented 1 year ago

I see. You are using Unimatch as your baseline... Our papers are focusing on different directions to improve the performance. Mixing these components may not get better results as you expected.

Hugo-cell111 commented 1 year ago

Get it. The provided result in Tab.4 of HAGSEG fully solves my problem. It is indeed a persuasive result. Thanks a lot! BTW, do you submit your paper of ICLR 2023 or 2024? Since I haven't found any corresponding thesis in ICLR 2023 openreview.

Hugo-cell111 commented 1 year ago

What's the meaning of "mixing these components"? Since I am a beginner in this area, I will greatly appreciate it if you could provide some suggestions. Thanks!

ZhenZHAO commented 1 year ago

Hi... my initial submission to ICLR 2023 is rejected. Since I thought reviewers did not get my points, I revised the draft and highlight the "instance and model". In the whole 2023, we did more explorations on semi-supervised learning, but it is not easy for us to provide a consistent better solution than augseg, unimatch. Especially for me, I did not have new output this year, focusing more on LLMs.

Sorry, "mixing these components", is not a good saying. These works are focusing on different directions, they can be useful compared to the baseline, but direct combining all these techs can not produce a consistently better resutls. Especially on cityscapes. BTW, it is not hard to further improve the performance on Pascal VOC.

I just checked the ICCV accpected papers, kinda frustrated on this topic...... any suggestions from you?

Hugo-cell111 commented 1 year ago

Quite agree with your opinion. In fact, I want to switch to another research area but since it is a huge sunk cost, as well as considering my graduation requirement, I have to go on researching in this area. But if I have finished my current work, I will also glad to switch to LLM, since there isn't any paper analysing the relationship between SSL and LLM. Sorry I can't provide any constructive advice, but I think maybe designing the plug-and-play component may work in adapting LLM to downstream dataset especially some sophisticated dataset such as Cityscapes.