Math for continuous X discrete state

[jlperla]

Add in a new major section to the derivation document where we will derive a continuous X discrete states.

• [x] First, put in a "notation" section.
  • Let the index of the discrete state be n = 1, ... N and the discrete operator for the stochastic process when in those states be L^n.
  • Let the intensity matrix of the stochastic process for the markov chain of the discrete state be Q.
  • We will assume that the discrete and continuous stochastic processes are independent, which will simplify the setup.
  • Add in an example, where N= 2 and there is a diffusion process with different drifts depending on the discrete state. That is, the continous processes would evolve as

    d X_{1t} = mu_1 dt + sigma d W_t
    d X_{1t} = mu_2 dt + sigma d W_t

    subject to reflecting boundaries at x_min and x_max.

and the intensity matrix of the markov chain is

Q = [-q_1 q_1
         q_2 -q_2]

With this, the payoffs were just the pi(x) then the stationary bellman equation discounted at rate r would be the system

r v_1(x) = pi_1(x) + mu_1 v_1'(t,x) + sigma^2/2 v_1''(t,x) + q_1 (v_2(x) - v_1(x))
r v_2(x) = pi_1(x) + mu_2 v_2'(t,x) + sigma^2/2 v_2''(t,x) + q_2 (v_1(x) - v_2(x))

Point out that if we stack the v in the discrete grid x_0 to x_M as

v_bar = [v_1(x_0)
           ...
           v_2(x_{M+1})  in R^(2(M+1))
pi = [pi_1(x_1)
            ...
            pi_1(x_M)
            pi_2(x_1)
            ....
            pi_2(x_M)] in R^(2M)

Then we can discretize this as an extension operator

L v_bar = pi

with 4 total equations in the boundary conditions.

v_1'(x_min) = 0
v_1'(x_max) = 0
v_2'(x_min) = 0
v_2'(x_max) = 0

(Resolved by ba99afb)

• [x] After you write down the example, write the more generic form of this given extension operators L_1, ... L_N for the continuous dimensions and the Q intensity matrix. The discretization of the joint process with M points on the interior is then a matrix of size an (M N) by ((M+2) N)

What does this look like? Take the n'th row and it looks something like this (I think)

[q_1 * banded_I     ...     q_(n-1) *banded_I      L_n-q_n * banded_I     q_(n+1) * banded_I     ....   q_N * banded_I]

And this row is of size M by (M+2)*N Where the banded_I is some sort of banded matrix with a band of ones at the diagonal or slightly off of it. (Resolved by 8d540dc)