Predictive Information

[dglmoore]

Implement predictive information in line with Bialek, Nemenman and Tishby [1, 2]. The excess entropy of Feldman and Crutchfield [3, 4].

Predictive information, as defined by Bialek, et. al., is the mutual information between a finite past and the semi-infinite future. Our implementation will use finite past and futures.

Note that we can think of active information as a special case of predictive information.

Proposed API

EXPORT double inform_predictive_info(int const *series, size_t n, size_t m, int b, size_t kpast,
    size_t kfuture, inform_error *err);
EXPORT double *inform_predictive_info(int const *series, size_t n, size_t m, int b, size_t kpast,
    size_t kfuture, double* ee, inform_error *err);

Example Usage

#define N 9

int xs[N] = {0,0,1,0,0,1,0,0,1};
inform_error err = INFORM_SUCCESS;
inform_predictive_info(xs, 1, N, 2, 3, 3, &err); // == 1.5

Thoughts on Implementations

We might consider implementing excess entropy in terms of predictive information, e.g.

double inform_excess_entropy(int const *series, size_t n, size_t m, int b, size_t k,
    inform_error *err)
{
    return inform_predictive_info(series, n, m, b, k, k, err);
}

[dglmoore]

Pull request #43 closed this issue.