Closed xjing76 closed 3 years ago
xjing76
commented
3 years ago Looking through the Theano graph of how we computed the log likelihood plot. I think there are a few places we are reallocating the space for sub distributions.
SwitchingProcess I am not sure if we are actually allocating space for logp or we are still broadcasting. However, this seems like a place where we can improved by reusing a shared variables for logp within SwitchingProcess
xjing76
commented
3 years ago Sum{axis=[0], acc_dtype=float64} [id A] ''
|InplaceDimShuffle{x,0} [id B] ''
|AdvancedIncSubtensor{inplace=False, set_instead_of_inc=True} [id C] ''
|AdvancedIncSubtensor{inplace=False, set_instead_of_inc=True} [id D] ''
| |Alloc [id E] ''
| | |TensorConstant{-inf} [id F]
| | |TensorConstant{30} [id G]
| |Elemwise{switch,no_inplace} [id H] ''
| | |Elemwise{mul,no_inplace} [id I] ''
| | | |InplaceDimShuffle{x} [id J] ''
| | | | |TensorConstant{1} [id K]
| | | |Elemwise{mul,no_inplace} [id L] ''
| | | |InplaceDimShuffle{x} [id M] ''
| | | | |TensorConstant{1} [id N]
| | | |Elemwise{eq,no_inplace} [id O] ''
| | | |AdvancedSubtensor [id P] ''
| | | | |TensorConstant{[ 0 0 ..0 0 0]} [id Q]
| | | | |Elemwise{eq,no_inplace} [id R] ''
| | | | |Elemwise{Cast{int32}} [id S] ''
| | | | | |S_t ~ DiscreteMarkovChain [id T]
| | | | |InplaceDimShuffle{x} [id U] ''
| | | | |TensorConstant{0} [id V]
| | | |AdvancedSubtensor [id W] ''
| | | |Elemwise{mul,no_inplace} [id X] ''
| | | | |InplaceDimShuffle{x} [id Y] ''
| | | | | |TensorConstant{0} [id Z]
| | | | |Alloc [id BA] ''
| | | | |TensorConstant{1.0} [id BB]
| | | | |Subtensor{int64} [id BC] ''
| | | | |Shape [id BD] ''
| | | | | |TensorConstant{[ 0 0 ..0 0 0]} [id Q]
| | | | |Constant{0} [id BE]
| | | |Elemwise{eq,no_inplace} [id R] ''
| | |InplaceDimShuffle{x} [id BF] ''
| | | |TensorConstant{0} [id BG]
| | |InplaceDimShuffle{x} [id BH] ''
| | |TensorConstant{-inf} [id BI]
| |Elemwise{eq,no_inplace} [id R] ''
|Elemwise{switch,no_inplace} [id BJ] ''
| |Elemwise{mul,no_inplace} [id BK] ''
| | |Elemwise{eq,no_inplace} [id BL] ''
| | | |AdvancedSubtensor [id BM] ''
| | | | |Elemwise{mul,no_inplace} [id BN] ''
| | | | | |InplaceDimShuffle{x} [id BO] ''
| | | | | | |TensorConstant{9.0} [id BP]
| | | | | |Alloc [id BQ] ''
| | | | | |TensorConstant{1.0} [id BR]
| | | | | |Subtensor{int64} [id BS] ''
| | | | | |Shape [id BT] ''
| | | | | | |TensorConstant{[ 0 0 ..0 0 0]} [id Q]
| | | | | |Constant{0} [id BU]
| | | | |Elemwise{eq,no_inplace} [id BV] ''
| | | | |Elemwise{Cast{int32}} [id S] ''
| | | | |InplaceDimShuffle{x} [id BW] ''
| | | | |TensorConstant{1} [id BX]
| | | |InplaceDimShuffle{x} [id BY] ''
| | | |TensorConstant{0} [id BZ]
| | |Elemwise{eq,no_inplace} [id CA] ''
| | |AdvancedSubtensor [id CB] ''
| | | |TensorConstant{[ 0 0 ..0 0 0]} [id Q]
| | | |Elemwise{eq,no_inplace} [id BV] ''
| | |InplaceDimShuffle{x} [id CC] ''
| | |TensorConstant{0} [id CD]
| |InplaceDimShuffle{x} [id CE] ''
| | |TensorConstant{0} [id CF]
| |Elemwise{switch,no_inplace} [id CG] ''
| |Elemwise{mul,no_inplace} [id CH] ''
| | |Elemwise{mul,no_inplace} [id CI] ''
| | | |InplaceDimShuffle{x} [id CJ] ''
| | | | |TensorConstant{1} [id CK]
| | | |Elemwise{mul,no_inplace} [id CL] ''
| | | |InplaceDimShuffle{x} [id CM] ''
| | | | |TensorConstant{1} [id CN]
| | | |Elemwise{ge,no_inplace} [id CO] ''
| | | |AdvancedSubtensor [id BM] ''
| | | |InplaceDimShuffle{x} [id CP] ''
| | | |TensorConstant{0} [id CQ]
| | |Elemwise{mul,no_inplace} [id CR] ''
| | |InplaceDimShuffle{x} [id CS] ''
| | | |TensorConstant{1} [id CT]
| | |Elemwise{ge,no_inplace} [id CU] ''
| | |AdvancedSubtensor [id CB] ''
| | |InplaceDimShuffle{x} [id CV] ''
| | |TensorConstant{0} [id CW]
| |Elemwise{sub,no_inplace} [id CX] ''
| | |Elemwise{sub,no_inplace} [id CY] ''
| | | |Elemwise{switch,no_inplace} [id CZ] ''
| | | | |Elemwise{eq,no_inplace} [id DA] ''
| | | | | |AdvancedSubtensor [id BM] ''
| | | | | |InplaceDimShuffle{x} [id DB] ''
| | | | | |TensorConstant{0} [id DC]
| | | | |Elemwise{switch,no_inplace} [id DD] ''
| | | | | |Elemwise{eq,no_inplace} [id DE] ''
| | | | | | |AdvancedSubtensor [id CB] ''
| | | | | | |InplaceDimShuffle{x} [id DF] ''
| | | | | | |TensorConstant{0} [id DG]
| | | | | |InplaceDimShuffle{x} [id DH] ''
| | | | | | |TensorConstant{0.0} [id DI]
| | | | | |InplaceDimShuffle{x} [id DJ] ''
| | | | | |TensorConstant{-inf} [id DK]
| | | | |Elemwise{mul,no_inplace} [id DL] ''
| | | | |AdvancedSubtensor [id CB] ''
| | | | |Elemwise{log,no_inplace} [id DM] ''
| | | | |AdvancedSubtensor [id BM] ''
| | | |gammaln [id DN] ''
| | | |Elemwise{add,no_inplace} [id DO] ''
| | | |AdvancedSubtensor [id CB] ''
| | | |InplaceDimShuffle{x} [id DP] ''
| | | |TensorConstant{1} [id DQ]
| | |AdvancedSubtensor [id BM] ''
| |InplaceDimShuffle{x} [id DR] ''
| |TensorConstant{-inf} [id DS]
|Elemwise{eq,no_inplace} [id BV] ''
I am not sure if I understand right. But I think we would like to duplicate the computation graph for each state. My attempt here However, it seems like when we are cloning the node. We are probably evaluating the operation the result of the clone is not a node but a tensor variable.
(Pdb) theano.printing.debugprint(dependent_rv.logp_elemwiset.clone())
<TensorType(float32, vector)> [id A]Here I get the traversing function to be conditioned on SwitchingProcess. And get the location of the alloc without traversing.
However, I think I am still failing the FFB unit test. The states that we are getting is not the state that we suppose to get. If @brandonwillard you can review this portion that would be great, as I am not confident that it is the right way to set the S_t. What I am trying to do is to set the St to be all 0 and all 1 for the two states just like in the original setup. However, the seems like for every run of ffb.step() the result for states are different, which makes me questions if this is right.
-> assert np.array_equal(res["S_t"], poiszero_sim["S_t"])
(Pdb) poiszero_sim["S_t"]
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 1, 1, 0, 0, 0, 0, 0])
(Pdb) ffbs.step(test_point)['S_t']
array([0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
0, 0, 0, 1, 1, 1, 0, 1])
(Pdb) ffbs.step(test_point)['S_t']
array([0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 0, 0,
0, 0, 1, 1, 0, 0, 0, 1])
(Pdb) ffbs.step(test_point)['S_t']
array([1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0,
1, 0, 0, 1, 0, 0, 1, 1])
Attempts to improve FFB by storing the likelihood series within sampler.
I feel like this is probably not as simple as this but hopefully is this the right idea?
If
log_lik_fnis subject to change across each step then this would fail.closes #76