search

srush / annotated-s4

Implementation of https://srush.github.io/annotated-s4
https://srush.github.io/annotated-s4
MIT License
468 stars 60 forks source link

Discretization Reference #58

Open successar opened 2 years ago

successar commented 2 years ago

Thanks for the great blog post, It was quite helpful for keeping track of what's happening in the paper.

Currently in the section "Discrete-time SSM: The Recurrent Representation" in the Annotated S4 page, you are linking to wikipedia page for bilinear transform. One thing I might suggest is to also link to the page for midpoint method for ODE discretization (since that is exactly same as bilinear transform in this case; I also feel ML community tend to know about Euler Method of ODE discretization and Midpoint method is just an extension of that -- Reading the bilinear transform page was not as informative for me on how it led to the given discretization). The method can be found at https://en.wikipedia.org/wiki/Midpoint_method

srush commented 2 years ago

Thanks, this is a thoughtful point. I will see if I can add some more text and examples here. Discretization is pretty important for these models.