NickGeneva
commented
2 years ago Hi @RuiGangzai ,
Those are both great questions that I don't have complete answers for. However here was my experience from playing around with deep Koopman.
For a new system, how to design the weight of each item in the loss function?
Indeed this is tricky, and usually, I found weighting the auto-encoder loss higher than the Koopman loss component was important. But this is entirely empirical. What really matters is controlling the number of time-steps you predict in a single loss (so how many Koopman matrix multiplications you apply). I found that you always want to start with just 1-2 steps and maybe slowly increase the number of steps as the model learns.
In addition, how to design the number of hidden layers and the number of neurons？
Not too much thought went into this for this work. Basically just running the model and testing a few different configurations to see what works best. Ideally, you can just specify a large enough network that is somewhat over-parameterized and the optimization will produce something that works. For example, I may just randomly start with a 4-layer model with 256 neuron layers because these are just nice numbers that work on the hardware I am training on. At the end of the day, this is a problem that all DL practitioners need to make a judgment call for any problem.
RuiGangzai
Hi @RuiGangzai
Sorry for the late response. When I was working on this I found that this form seemed to work better for the system in the blog but this doesn't necessarily extrapolate to other systems. I would suggest starting with a fully parameterized matrix and then slowly adding constraints to it empirically. This could be a block diagonal form, symmetric properties, etc. Typically I have found a fully Koopman matrix tends to overfit, finding high-frequency modes.
In my more recent paper, on page 10, there's some references to different deep Koopman work with different biases on the Koopman matrix.
https://arxiv.org/pdf/2010.03957.pdf