search

pyro-ppl / funsor

Functional tensors for probabilistic programming
https://funsor.pyro.ai
Apache License 2.0
234 stars 20 forks source link

Student t filters and processes #190

Open fritzo opened 5 years ago

fritzo commented 5 years ago

While Gaussian models are flexible and tractable, real data often exhibits noise with heavier tails than Gaussian. A popular model for robust estimation is Student's t distribution, e.g. as used in Slawek Smyl's models. Like the Gaussian, Student's t filters [1,2,3] and processes [4,5] with shared dof parameter are composable in exact inference. Moreover approximate algorithms allow for differing dof parameters in different parts of the model [3].

References

  1. Michael Roth, Emre Ozkan and Fredrik Gustafsson (2013) A student's t filter for heavy tailed process and measurement noise
  2. Christian Rover (2013) A Student-t based filter for robust signal detection
  3. Filip Tronarp, Toni Karvonen, Simo Särkkä (2019) Student’s t-Filters for Noise Scale Estimation
  4. Amar Shah, Andrew Gordon Wilson, Zoubin Ghahramani (2014) Student-t Processes as Alternatives to Gaussian Processes
  5. Brendan D. Tracey, David H. Wolpert (2018) Upgrading from Gaussian Processes to Student's-T Processes
  6. A Solin, S Särkkä - Artificial Intelligence and Statistics, 2015 State space methods for efficient inference in Student-t process regression
fritzo commented 5 years ago

cc @martinjankowiak

martinjankowiak commented 5 years ago

ah, thanks, i was unaware of some of those. added [6]

fritzo commented 5 years ago

@fehiepsi Do you have time to look into a pyro.ops.studentt analogous to pyro.ops.gaussian? I think a pyro.distributions.StudentTHMM would be a very competitive time series model if we can make the math work (say with common dof for all transition and emission distributions).

Note I'm not sure how much is possible, e.g. maybe we can only support univariate Student t.

fehiepsi commented 5 years ago

@fritzo I have some discussions with @martinjankowiak previously about MVStudentT. Hope that it already settles all the math. I will see if it works in a few days (currently I am working on square root version of Gaussian).

martinjankowiak commented 5 years ago

aah unfortunately all those conversations were in the context of variational interference (and so not directly applicable)

On Mon, Aug 19, 2019, 6:04 PM Du Phan notifications@github.com wrote:

@fritzo https://github.com/fritzo I have some discussions with @martinjankowiak https://github.com/martinjankowiak previously about MVStudentT. Hope that it already settles all the math. I will see if it works in a few days (currently I am working on square root version of Gaussian).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pyro-ppl/funsor/issues/190?email_source=notifications&email_token=AFIUZAXN627SPTOQDFZRGRLQFLABTA5CNFSM4IKN22N2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4TOSNY#issuecomment-522643767, or mute the thread https://github.com/notifications/unsubscribe-auth/AFIUZAS34YPD6ZEXGETW3STQFLABTANCNFSM4IKN22NQ .

fehiepsi commented 5 years ago

This looks non-trivial but seems doable and interesting. I'll take care of implementing studentt but it might take a bit of time for me to settle all the math (especially to see if we can derive an information form for the filter in reference [3]).

fritzo commented 4 years ago

I think we could do this with an auxiliary variable reparameterizer like pyro.poutine.reparam. In funsor I believe this would be a reparam interpretation that could e.g. transform a StudentT or Stable to a Joint with other variables. I doubt we yet have the machinery for an end-to-end heavy-tailed HMM example, but that could probably be done in ~1 week of coding. See Pyro's Forecasting II example.