search

Lafite-Yu / MTTrans-OpenSource

10 stars 1 forks source link

Questions on momentum updator #5

Open ilhoon23 opened 1 year ago

ilhoon23 commented 1 year ago

If you use the default values of momentum=0.999, interval=1, warm_up=100, momentum = min(self.momentum, 1 - (1 + self.warm_up) / (self.curr_step + 1 + self.warm_up)) gets the value of self.momentum when curr_step become 100900, and this much step seems to be too much considering the size of the dataset and the batch size. Can i get some tips on getting the mAP value of 35.843, where only mean teacher and shared QE is used?

Lafite-Yu commented 10 months ago

If you use the default values of momentum=0.999, interval=1, warm_up=100, momentum = min(self.momentum, 1 - (1 + self.warm_up) / (self.curr_step + 1 + self.warm_up)) gets the value of self.momentum when curr_step become 100900, and this much step seems to be too much considering the size of the dataset and the batch size. Can i get some tips on getting the mAP value of 35.843, where only mean teacher and shared QE is used?

The most important tip for the cityscapes dataset is to load the pretrained weights from the SFA. Training our model from scratch did not get satisfying results by us on this dataset (however, on the other two datasets, the results are pretty good). And actually, you can simply surpass the performance reported in the SFA paper by loading and continuing to train SFA for some more epochs...