Add features from Ebisu 2.1

kirianguiller commented 3 years ago

Hey everyone,

This morning I did some testing on the updateRecall function, and I was really surprised on the behavior. The function outputs are both not logical and different from the python API.

Javascript

Input

Here the ebisuIssue.js script :

// ebisuIssue.js
import * as ebisu from 'ebisu-js';

console.log(ebisu.updateRecall([2, 2, 1], 1, 1, 1));
console.log(ebisu.updateRecall([2, 2, 1], 1, 1, 2));
console.log(ebisu.updateRecall([3, 3, 1], 1, 1, 1));
console.log(ebisu.updateRecall([3, 3, 24], 1, 1, 24));

output

and here is the output :

[ 2.999999999999825, 1.9999999999999112, 1 ]
[ 1.1246051461015583, 2.079525890708407, 3.0652051705190964 ]
[ 4, 2.9999999999999867, 1 ]
[ 4, 2.9999999999999867, 24 ]

Problems

for a tnow=halftime, its like if there is no change at all, whereas it should be
for tnow > halftime, the changes in alpha and beta are not symetrical, whereas in the python library, if alpha and beta are equal before updating the model, they stay equal after too

Python

For comparaison, here what we should find with python

input

print(ebisu.updateRecall([2, 2, 1], 1, 1, 1))
print(ebisu.updateRecall([2, 2, 1], 1, 1, 2))
print(ebisu.updateRecall([3, 3, 1], 1, 1, 1))
print(ebisu.updateRecall([3, 3, 24], 1, 1, 24))

output

(2.021907084012532, 2.0219070840133453, 1.4244289008985422)
(2.0309464487893827, 2.030946448789381, 1.8442887702247592)
(3.0321060934248467, 3.032106093424845, 1.2681019905723685)
(3.0321060934248467, 3.032106093424845, 30.434447773736842)

Their might be a problem in the code somewhere

PS : predictRecall() is behaving the same on both python and javascript, so the problem is probably only in the code of updateRecall()

fasiha commented 3 years ago

Ahh, right, so the issue is that this JavaScript repo implements Ebisu v2.0 (https://github.com/fasiha/ebisu.js/blob/gh-pages/package.json#L3) and doesn't yet have the updates that the Python version of Ebisu introduced in 2.1, see https://github.com/fasiha/ebisu/blob/gh-pages/CHANGELOG.md#210-soft-binary-quizzes-and-halflife-rescaling

noisy quizzes (q0 and q1),
rescaleHalflife, and
updateModel automatically rebalances to the new halflife.

So this repo (and ebisu-java) still works fine, just without these these features. Forgive me, this isn't the first time I release a new version of Python Ebisu and then it takes me a few weeks/months to release a new JS version 😭.

Do you need the new 2.1 features? If so I can prioritize updating this repo to 2.1.

kirianguiller commented 2 years ago

Indeed, these 2.1 features would be useful for our project.

I was thinking about, if we use ebisujs 2.0 at the moment in our app, our t from [a,b;t] will not represent the half-life. So I was concern about when version 2.1 is released (or 3.X directly), would it be possible to "convert" 2.0 models to 2.1 models (that directly have half-time in it). It seems that it's what "tback" could do from updateRecall(prior, successes, total, tnow, rebalance=True, tback=None, q0=None) (first would need to compute the halftime with modelToPercentileDecay and then force it in updateRecall by specifying the tback as the halftime (and by having rebalance=True, which is the default behavior anyway atm)).

Is it the best way for converting 2.0 to 2.1 models ? Thanks !

fasiha commented 2 years ago

would it be possible to "convert" 2.0 models to 2.1 models

Yes @kirianguiller! An easy way to do this with 2.1 is to use the rescaleHalflife function which will by default return the model for the halflife. (It's intended to rescale that halflife but the default scaling is 1.0.

Converting version 2.x to 3 models will be interesting.

There's of course an easy way to do it: just convert the Ebisu 2.x probability distribution on recall probability at the halflife to Ebisu 3's probability distribution on the halflife itself (I'll provide an API to do this).
But the nice thing about Ebisu 3 is that it uses a list of past quizzes to estimate the boost (Anki's ease factor) that the halflife grows by after each review. If you can create such a list of past quizzes (i.e., if you've stored past quiz time and result info in your database), you can run the Ebisu 3 update step to get an accurate estimate.

Hope this helps! My goal is to finish adding unit tests to Ebisu 3 before New Year's!

fasiha commented 1 year ago

Fixed by #19, just published v2.1. Sorry for the delay!

fasiha / ebisu.js