missing reference

[2prime]

Thanks for your list of neuralode papers, I'm also working on using deep learning to approximate dynamic systems. I'm writing this email to introduce my recent work

Our work starts from bidging deep architects are numerical schemes: Yiping Lu, Aoxiao Zhong, Quanzheng Li, Bin Dong. "Beyond Finite Layer Neural Network:Bridging Deep Architects and Numerical Differential Equations" Thirty-fifth International Conference on Machine Learning (ICML), 2018 （On arxiv 2017.11，iclr workshop track paper）

using deep networks to find out the PDE behind data Zichao long, Yiping Lu, Xianzhong Ma, Bin Dong. "PDE-Net:Learning PDEs From Data",Thirty-fifth International Conference on Machine Learning (ICML), 2018(equal contribution)

We also show that Using ODE cam help design optimization method for neural networks【especially adversarial training！we are 5 times faster！ Dinghuai Zhang, Tianyuan Zhang,Yiping Lu, Zhanxing Zhu, Bin Dong. "You Only Propagate Once: Painless Adversarial Training Using Maximal Principle." (equal contribution) Submitted. arXiv preprint:1905.00877 （Neurips2019）

We also aim to use ODE to understand NLP/sequence modeling Yiping Lu, Zhuohan Li, Di He, Zhiqing Sun, Bin Dong, Tao Qin, Liwei Wang, Tie-yan Liu "Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View." (*equal contribution) Submitted. arXiv preprint:1906.02762

to learn early stopping for image restoration using optimal control and reinforcement learning Xiaoshuai Zhang, Yiping Lu, Jiaying Liu, Bin Dong. "Dynamically Unfolding Recurrent Restorer: A Moving Endpoint Control Method for Image Restoration" Seventh International Conference on Learning Representations(ICLR) 2019(*equal contribution)

welcome to my homepage to explore more https://web.stanford.edu/~yplu/ we will have more related works recently

Also I want to figure out some missing reference for theory paper Thorpe M, van Gennip Y. Deep limits of residual neural networks[J]. arXiv preprint arXiv:1810.11741, 2018. A mean-field optimal control formulation of deep learning arxiv 2018

For optimization using ODE papers you can see Maximum Principle Based Algorithms for Deep Learning JMLR2018， arxiv 2017 An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks ICML2018

And the first paper introducing the ode idea is Weinan E. A proposal on machine learning via dynamical systems[J]. Communications in Mathematics and Statistics, 2017, 5(1): 1-11.

[Zymrael]

Thank you for the list of additional references! If you find / publish more feel free to share. I will add these asap.

[dandelin]

Let me introduce our paper

Kim, Wonjae, and Yoonho Lee. "Learning Dynamics of Attention: Human Prior for Interpretable Machine Reasoning." arXiv preprint arXiv:1905.11666 (2019). (NeurIPS 2019)

We modeled the update of attention logits as neural ODEs to build more interpretable VQA model. Take a look 😄.

[Zymrael]

@dandelin I've been made aware of your interesting work by others colleagues, I'll add it after I finish taking a deeper look at the paper. We had some ideas about incorporating attention and neural ODEs ourselves but had to wrap up other research first. Perhaps a collaboration waiting to happen?

[dandelin]

@Zymrael Absolutely! great to hear that. You can contact me via what you want 👍 . https://wonjae.kim/