In order to compute the likelihood (a lower bound for it to be exact) for any LDA model one needs to compute the expectation step in order to maximize according to the variational parameters.
However, in order to compute the expectation step one needs not compute the likelihood.
Thus I propose the following convention which unfortunately cannot be enforced via an interface.
We shall introduce to all expectation steps a constructor parameter Scalar compute_likelihood in the range [0, 1] that means emit an ExpectationProgressEvent containing likelihood with probability compute_likelihood.
Currently likelihood is always computed for the SupervisedEStep and others. For ApproximateSupervisedEStep likelihood is computed only if the boolean compute_likelihood is true. I propose a universal behavior as stated above.
Positive values, NaNs and INFs can all be used to signal that no likelihood was computed. This means that all expectation steps shall emit an ExpectationProgressEvent containing the above invalid values with probability 1-compute_likelihood.
In order to compute the likelihood (a lower bound for it to be exact) for any LDA model one needs to compute the expectation step in order to maximize according to the variational parameters.
However, in order to compute the expectation step one needs not compute the likelihood.
Thus I propose the following convention which unfortunately cannot be enforced via an interface.
We shall introduce to all expectation steps a constructor parameter
Scalar compute_likelihoodin the range[0, 1]that means emit anExpectationProgressEventcontaining likelihood with probabilitycompute_likelihood.Currently likelihood is always computed for the
SupervisedEStepand others. ForApproximateSupervisedESteplikelihood is computed only if the booleancompute_likelihoodis true. I propose a universal behavior as stated above.Positive values, NaNs and INFs can all be used to signal that no likelihood was computed. This means that all expectation steps shall emit an
ExpectationProgressEventcontaining the above invalid values with probability1-compute_likelihood.What do you think @paschalidoud ?