search

SCNU-RISLAB / MF-MOS

This repo is the offical PyTorch implementation of MF-MOS.
GNU General Public License v3.0
42 stars 4 forks source link

Inquiry About Training Settings #3

Open pointcloud1999 opened 2 months ago

pointcloud1999 commented 2 months ago

Hello,

During my training, I observed that in the ddp_mos_coarse_stage.yml file, the learning rate (lr) is set to 0.002, whereas in the mos_coarse_stage.yml file, it is set to 0.008. Could you please explain the reason for these different settings? Additionally, I noticed that the initial learning rate mentioned in the paper is 0.01. Which learning rate should I follow for my experiments? What are the differences between these two yml files?

Thank you.

Terminal-K commented 2 months ago

Hi @pointcloud1999, lr is different in ddp_mos_coarse_stage.yml and mos_coarse_stage.yml because we keep the lr positively correlated with the batch size, meaning the learning rate for per batch size is fixed. Besides, the inconsistency of lr in the files and the lr reported in the paper may be due to a small mistake, but it is unlikely to have a significant impact on performance in my opinion. I suggest conducting experiments based on the lr in the code first. However, if you have extra spare time, you can also perform comparative experiments to test its impact on performance [ I do not have a suitable experimental environment currently :( ]. Thanks for your keen observation!