search

microsoft / Semi-supervised-learning

A Unified Semi-Supervised Learning Codebase (NeurIPS'22)
https://usb.readthedocs.io
MIT License
1.37k stars 182 forks source link

som question about get_cosine_schedule_with_warmup #206

Closed wenyuhaokikika closed 6 months ago

wenyuhaokikika commented 8 months ago

Hello , thanks for Semi-supervised-learning,

semilearn/core/utils/build.py:217:def get_cosine_schedule_with_warmup

I want to know why num_cycles set 7. / 16，have some referance？

And I checked the original algorithm of the cosine Annealing algorithm. It seems to be slightly different from the version in the code. Is it possible that TorchSSL does not use the original algorithm but a self-set algorithm with a reduced learning rate? or I found a wrong reference paper?

reference

and TorchSSL is :

image

Looking forward for your reply, thank you~~~

### Tasks
github-actions[bot] commented 6 months ago

Stale issue message

wenyuhaokikika commented 6 months ago

why close?

Hhhhhhao commented 6 months ago

why close?

Seems to be a stale close. The num_cycle of 7/16 is set as a default since FixMatch paper. You can find more details there.