DP6 / Marketing-Attribution-Models

Python Class created to address problems regarding Digital Marketing Attribution.
https://dp6.github.io/Marketing-Attribution-Models
Apache License 2.0
302 stars 80 forks source link

[BUG] Markov model attribution may sometimes not work as intended due to the way it is computed #41

Closed taksqth closed 2 years ago

taksqth commented 2 years ago

Sometimes when calling mam.attribution_markov() the procedure may produce a warning stating a Singular Matrix Error and when that happens the attributed values are not reliable (it is possible that negative numbers are produced, which doesn't make much sense in this context). This happens due to way that the removal effect is computed, which requires an eigenvector factorization to get an infinite power for the transition matrix. This eigenvector matrix may not be inversible, which leads to the error. The exact kind of scenarios where this may happen are not clear and may even be due to numeric errors, as they usually seem to happen when some channels have much less interactions than others.

Fortunately, further research on the topic is not needed, because what we want is the probability that the user will eventually reach the (absorbing) conversion state from the starting state from our constructed transition matrix. Computing the infinite power was a first (naive) implementation to get that, but it is possible to compute those absorving properties via the following code (as described in the wikipedia article):


Q = matrix[:-2,:-2] # Those indices follow from the construction, where we have the conversion 
R = matrix[:-2,-2:] # and non-conversion states as the last 2

N = np.linalg.inv(np.identity(len(Q)) - Q)
return (N @ R)[0,1] # We also assume the first row represents the starting state

The matrix N is called the fundamental matrix and it can be proven that it will always exist (Theorem 2.2.2 in [1]), given the properties of how we expect the conversion matrixes to be constructed in MAM.

We should, therefore, change the implementation to use this new method.

[1] Kassir, A. (2018). Absorbing Markov Chains with Random Transition Matrices and Applications. UC Irvine. ProQuest ID: Kassir_uci_0030D_15141. Merritt ID: ark:/13030/m50k7645. Retrieved from https://escholarship.org/uc/item/52h7q78h