Closed Datseris closed 1 year ago
@kahaaga what's up with the signature entropy([e::Entropy,] est::EntropyEstimator, x)
, why does it exist? How can a user specify an entropy definition and an entropy estimator? The estimator by definition computes a specific version of entropy, since its an estimator. If it wasn't an estimator of an entropy directly, then surely it must be computing a probability density, and hence it isn't an entropy estimator, but a probability estimator.
Are we complicating things too much here? Why would we even care to allow this. I say we should have entropy(est::Estimator, x)
and make clear that this is a completely separated method.
@kahaaga Unfortunately I have to change FixedRectangularBinning
in this PR. It is a nightmare to allow it to both have "actuall fixed" size by providing εmin/max for each dimension, but also allow it to deduce the dimensions by itself and hence having arbitrary dimension. I'll make a different convenience method for that. For FixedRectangularBinning
the input dimension must always be given clearly so that the struct always has well defined outcome space without requiring the input.
@kahaaga what's up with the signature entropy([e::Entropy,] est::EntropyEstimator, x), why does it exist? How can a user specify an entropy definition and an entropy estimator? The estimator by definition computes a specific version of entropy, since its an estimator. If it wasn't an estimator of an entropy directly, then surely it must be computing a probability density, and hence it isn't an entropy estimator, but a probability estimator. Are we complicating things too much here? Why would we even care to allow this. I say we should have entropy(est::Estimator, x) and make clear that this is a completely separated method.
I think we should keep the three-argument version. This is related to the answer I linked in #184. In short:
ProbabilitiesEstimator
s) and differential/continuous entropy (i.e. EntropyEstimator
s) allows seamless construction of plug-in estimators for any entropy-based information measure in CausalityTools. One method signature = one unified wrapper, not many wrappers.LeonenkoProzantoSavani <: EntropyEstimator
(which currently only exists on a dev branch of CausalityTools) can compute Shannon
, Renyi
and Tsallis
entropies (see screenshot), all based on the estimation of the same underlying integral. The alternative would be to define LeonenkoProzantoSavaniShannon
, LeonenkoProzantoSavaniTsallis
, LeonenkoProzantoSavaniRenyi
, which I think is much more messy, when the underlying estimators are identical, up to some scaling factor.Follow-up to the last post: The reason I wouldn't add separate LeonenkoProzantoSavaniShannon
, LeonenkoProzantoSavaniRenyi
, LeonenkoProzantoSavaniTsallis
estimators is the same reason I wouldn't add separate TransferOperatorTsallis
, TransferOperatorRenyi
, TransferOperatorShannon
probability estimators.
EntropyEstimator
s implicitly or explicitly estimate some integral (usually a density, but not always). This integral estimate is then plugged into some formula. In the case of the LeonenkoProzantoSavani
estimator, the formula for Shannon, Renyi and Tsallis entropy are different, both all contain the same integral, so the same estimator is used to compute all three quantities.
Unfortunately I have to change FixedRectangularBinning in this PR. It is a nightmare to allow it to both have "actuall fixed" size by providing εmin/max for each dimension, but also allow it to deduce the dimensions by itself and hence having arbitrary dimension. I'll make a different convenience method for that. For FixedRectangularBinning the input dimension must always be given clearly so that the struct always has well defined outcome space without requiring the input.
Ok, no problem. As long as the public API has an easy way to specify a fixed grid in multiple dimensions, the specifics of the underlying implementation doesn't really matter.
ok then I merge this. In a separate PR we should separate the documentation of the entropy with hte entropy estimators into a different documentation block. especially since you say they compute differential entropy.
Closes #185, closes #184
Implements the final version of the API which is:
encode/decode
.Also re-writes
FixedRectangularBinning
to have welldefined outcome space always.