akundaje
commented
6 years ago Good try.
Will be good to read some classic recent papers on curriculum learning to avoid forgetting. There are specific tricks to avoid that from happening involving playing with learning rates etc.
-Anshul.
On Mon, May 21, 2018 at 6:06 PM, annashcherbina notifications@github.com wrote:
[image: image] https://user-images.githubusercontent.com/5261545/40336920-3c4b3116-5d21-11e8-9c2e-639cbc0e1305.png
Performance on each negative set for each of the four phases in the curriculum:
[image: image] https://user-images.githubusercontent.com/5261545/40336946-64a00178-5d21-11e8-85cc-afd5ce5a5aff.png
So not where we want it to be yet... Next steps:
1.
In this first pass, I used a new negative set for each phase. In the next pass, I will mix in examples from prior phases so the model doesn't "forget" the patterns it learned in the previous phases. 2.
The accuracy on the positive examples seems to decrease across the phases. Will experiment with balanced batches in conjunction with data augmentation from point 1) to help the model learn positive examples.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kundajelab/gecco-variants/issues/7, or mute the thread https://github.com/notifications/unsubscribe-auth/AAI7ETHWDS6T8bW4s1enYLvM7eqlHK62ks5t02SQgaJpZM4UH0vk .
Performance on each negative set for each of the four phases in the curriculum:
So not where we want it to be yet... Next steps:
1) In this first pass, I used a new negative set for each phase. In the next pass, I will mix in examples from prior phases so the model doesn't "forget" the patterns it learned in the previous phases.
2) The accuracy on the positive examples seems to decrease across the phases. Will experiment with balanced batches in conjunction with data augmentation from point 1) to help the model learn positive examples.