The "increment entropy" paper defines a new OutcomeSpace that first embeds the input data, then encodes this data based on the difference between consecutive pairs of embedding vectors, including both sign information and magnitude information. This is an obvious new OutcomeSpace with corresponding encode/decode based on RectangularBinEncodings.
The method then estimated relative frequencies over the encoded symbols and plugs the estimated probabilities into an entropy formula.
The "increment entropy" paper defines a new
OutcomeSpace
that first embeds the input data, then encodes this data based on the difference between consecutive pairs of embedding vectors, including both sign information and magnitude information. This is an obvious newOutcomeSpace
with correspondingencode
/decode
based onRectangularBinEncoding
s.The method then estimated relative frequencies over the encoded symbols and plugs the estimated probabilities into an entropy formula.
We should implement the method.