search

quinngroup / CiliaRepresentation

A modular generative pipeline for understanding cilia appearance and dynamics
MIT License
0 stars 1 forks source link

Discussion: Beta scheduling #32

Open Micky774 opened 4 years ago

Micky774 commented 4 years ago

The overfit test got me thinking: what if we scheduled beta? If initially beta started off at 0, or at least low like .1, then throughout training leveled out at 1. I'm thinking that this may encourage the network to first learn meaningful patterns and features in the encoder, and then ramp up the task to also having a cohesive latent space. Thoughts?

idhumphrey commented 4 years ago

I think that before we implement this idea, it would be helpful to run experiment 10 to explore what initial beta we would want and then in that case, we could compare the results of having a scheduled beta to the best result from exp10.