search

DiffEqML / torchdyn

A PyTorch library entirely dedicated to neural differential equations, implicit models and related numerical methods
https://torchdyn.org
Apache License 2.0
1.36k stars 125 forks source link

Easy to Train Neural ODE #82

Closed pharringtonp19 closed 2 years ago

pharringtonp19 commented 3 years ago

Really enjoying the library so far --

I was wondering if there was any thoughts about implementing the regularization method introduced in Kelly 2020. I would be more than happy to assist in the implementation.

Thanks

Zymrael commented 3 years ago

Hi @pharringtonp19. We'd be glad to have that in the library. We're currently undergoing a major redesign of API and functionality in preparation for a v1.0 release. I'll ping you here once the API is stable enough that work on higher-level features such as regularization techniques is possible with minimal hassle on your end.

Zymrael commented 3 years ago

0.3.1 will go up later this week. There is still a lot happening around the numerical methods API but odeint, ODEProblem and NeuralODE should be stable enough for you to start. To implement regularizers, I would start with trying to reproduce some of their results on toyish examples in a notebook; after that we can think about how to best incorporate the method in the library (perhaps hiding it under some well designed abstraction grouping several regularizers, we'll see)

pharringtonp19 commented 3 years ago

I have a partial implementation of this now in JAX. Once I have it complete, I will see if I can write it up in torchdyn

regularized_neural_ode

Zymrael commented 3 years ago

Awesome! Let me know if you encounter difficulties.

Zymrael commented 2 years ago

Any news on this front @pharringtonp19? Would be cool to add this for our next small release.

pharringtonp19 commented 2 years ago

@Zymrael Through discussions with one of the authors on the paper, I realized that my implementation was not efficient. I can close this issue now and re-open it once I have a "complete" working version of this in jax. I hope to have it complete sometime over the winter break - Thanks