Open emanjavacas opened 3 years ago
From the last discussion,t we concluded to leave out the stuff coming from 3D party libraries (including the delayed stuff)
Delayer is not third party, I coded it :) And it's only with Delayer that Cosine outperforms significantly Adam :)
Could you pin point what is it that the Delayer class is achieving? I see a lot of new code involved in printing stuff and so on. Perhaps there is an easier way to add what the Delayer class contributes.
There is definitely. May I pr your PR ?
Ye, go ahead. Only I'd rather really keep it simple. For instance if there is that class that delays the step, we don't need to add a bunch of new functions to adapt the printing code for that class.
Check the last commit, that might solve the issues
Great !
Any chance you could quickly replicate the results for the combination that was on par with ReduceLROnPlateau but was faster? (No rush here)
On Mon, Dec 7, 2020 at 6:25 PM Thibault Clérice notifications@github.com wrote:
Great !
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/emanjavacas/pie/pull/79#issuecomment-740062144, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPIPI4DR6VURKQ7RURLHH3STUFYFANCNFSM4UQORVYQ .
-- Enrique Manjavacas
Running:
"lr_scheduler": "CosineAnnealingLR", // LR Scheduler to use: ReduceLROnPlateau,
"lr_scheduler_delay": 10,
"lr_scheduler_params": {
"T_max": 40,
"min_lr": 0.000001, // minimum learning rate
"lr_patience":7,
"factor":0.6,
},
You don't want me to rerun the 2x10 runs right ?
It works: ran once on my personal GPU for 67 epochs (higher parts of the box in https://github.com/emanjavacas/pie/issues/76#issue-757884805) with score within the box 97.61. (previous version of comment was using dev score, because I am tired and can't read)
Here is a draft for the lr_scheduler params (related to #78) and how to avoid breaking backwards compatibility. Could you use it to test the results of the cosineannealing that you got?