search

czekster / markov

Materials for book: "Markov Chains for programmers"
https://czekster.github.io/markov/
GNU General Public License v3.0
276 stars 20 forks source link

embedded DTMC #1

Closed jesuslop closed 2 years ago

jesuslop commented 2 years ago

Hi, I'm not managing to find the motivation or an intuitive route to the formula for the "embedded DTMC" of a CTMC in 3.2 pg. 31, apr. 22 ed. There is just the formula. Is it the stochastic matrix resulting of integrating the infinitesimal generator for some time? If it is so, how much time? A unit (a minute, and hour)? But if so it would be unit-choice dependent. Some extra light would be great. Any case, nice project, thanks for the effort!

czekster commented 2 years ago

Hi, don't know if it will improve your understanding, but try to see this PRISM slides (number 12) for more explanations on the Embedded DTMC: https://www.yumpu.com/en/document/read/47898048/continuous-time-markov-chains-prism

When you convert the CTMC to DTMC you no longer operate thinking about time (continuously), now you have a DTMC and it has discrete time 'evolution'. Please, let me know if this is sufficient. The book of Stewart (1995) may also help you, it's quite straightforward. Best,

jesuslop commented 2 years ago

Ok, that slide 12 sufficed me. The DTMC embeds in the CTMC timeline by abiding in each step a random time exponentially distributed with parameter the exit rate of the state it is in. Thanks!