Recalculate intervals for all cards in the deck when requestRetention or maximumInterval are changed

[Expertium]

Which module is related to your feature request? Scheduler

Is your feature request related to a problem? Please describe. Currently, even if the user changes requestRetention or maximumInterval, it will only affect future reviews, but won't affect past reviews and hence the current intervals of cards in the deck. Suppose user always had requestRetention = 0.9 and now he wants to set it to 0.8, but all the cards he has reviewed so far are scheduled using requestRetention = 0.9. Or maybe the user just started using FSRS and his cards aren't even scheduled according to FSRS. Or maybe the user changed parameters and wants to re-calculate the intervals using new parameters. Same goes for changing maximumInterval.

Describe the solution you'd like There should be a button (if possible) such that after pressing it the algorithm will re-calculate intervals for all cards in the current deck that have at least 1 review.

[L-M-Sherlock]

It is a useful feature. But it needs to build an add-on that could modify the database of Anki. And it requires that these cards have the stability in customData. I will try to develop it in the future.

[L-M-Sherlock]

I implement it in this add-on: FSRS4Anki Helper Could you give it a try?

[Expertium]

Ok. Also, I suggest you to add the link to github repository of FSRS to the add-on page on ankiweb. There is also something I wanted to ask - do you think it's possible to turn all of that code (scheduler + optimizer) into a single add-on with just a handful of buttons? Because in its current state, even if everything works smoothly, it's just too complex and not very user-friendly. Basically, I'm worried about mass adoption, and if mass adoption is something you are worried about as well, then the final version needs to be easy to install, work right out of the box with only a small number of buttons, and be as fool-proof as possible.

EDIT: Nope, doesn't work right now: Anki 2.1.55 (6944210f) Python 3.9.7 Qt 6.3.2 PyQt 6.3.1 Platform: Windows-10-10.0.19042 Flags: frz=True ao=True sv=3 Add-ons, last update check: 2022-10-01 23:42:21

Caught exception: Traceback (most recent call last): File "C:\Users\user\AppData\Roaming\Anki2\addons21\759844606__init.py", line 23, in a.triggered.connect(lambda b, did=did: fun(did)) File "C:\Users\user\AppData\Roaming\Anki2\addons21\759844606\init.py", line 100, in rescheduleFromDid reschedule(cids) File "C:\Users\user\AppData\Roaming\Anki2\addons21\759844606\init__.py", line 69, in reschedule if card.type != 2 or "s" not in json.loads(card.custom_data): File "json", line 346, in loads File "json.decoder", line 337, in decode File "json.decoder", line 355, in raw_decode json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[L-M-Sherlock]

There is also something I wanted to ask - do you think it's possible to turn all of that code (scheduler + optimizer) into a single add-on with just a handful of buttons?

It needs to implement the neural network in Anki's Rust backend. I didn't learn this programming language. So I don't know if it is possible to integrate the scheduler and optimizer into Anki. And it is difficult to turn all of that code (scheduler + optimizer) into a single add-on, because Anki's add-on can't use complicated neural network pylib.

[L-M-Sherlock]

EDIT: Nope, doesn't work right now

Thanks for the test. I am fixing it.

[Expertium]

Another question - the rescheduling add-on asks me to enter retention and maxinterval, but can't it read those from the code? I mean, I have a bunch of code for many decks, with different parameters and different max. intervals and stuff. It would be more convenient if the add-on went through the code and used that info, and if a particular deck (or a bunch of decks since I use deck_name.startsWith) doesn't have a deliberately specified set of parameters and retention and maxinterval, then just rely on global parameters (the ones at the top of the code)?

[L-M-Sherlock]

I update the add-on. You can try it now.

[L-M-Sherlock]

Another question - the rescheduling add-on asks me to enter retention and maxinterval, but can't it read those from the code?

I think it is not a frequent action to modify the parameters. And I am a noob in developing Anki's add-on. Maybe I will automate it in the next few versions.

[Expertium]

I think it is not a frequent action to modify the parameters.

That's not the issue, the issue is that I have to manually enter retention rate and maxinterval for like 7 different decks, which is kinda annoying. And I can't even tell if this add-on will use the specific parameters for each deck, or global parameters (I'm referring to the parameters of the DSR model, not retention and maxinterval). Anyway, I checked it and it seems to work, but since I don't know which parameters it used - global or deck-specific - I'm gonna restore my collection from a backup right now, just in case.

[L-M-Sherlock]

And I can't even tell if this add-on will use the specific parameters for each deck, or global parameters

This add-on doesn't use any other parameters. It only uses the stability of cards. And the stability is calculated by scheduler if the card has been reviewed when you enable FSRS. If the card hasn't been reviewed in FSRS, the stability is converted from the interval of card given by the built-in schedule.

[L-M-Sherlock]

That's not the issue, the issue is that I have to manually enter retention rate and maxinterval for like 7 different decks, which is kinda annoying.

It's annoying if you have a lot of decks. Maybe what you need is to monitor the requestRetention and maximumInterval in the custom scheduling code, and automatically update the intervals of cards when you modify these two parameters. But it requires the add-on to parse the code and react to the modification in live. I think Anki doesn't support it by far.

[Expertium]

I was talking about this:

// Default parameters of FSRS4Anki for global
var f_s = [0.4727,1.427];
var f_d = [1.0213,-0.3649,-2.0518,0.2074];
var s_w = [3.6427,-1.8115,-0.076,1.7659,1.384,-0.4759,0.7697,1.7983];
// The above parameters can be optimized via FSRS4Anki optimizer.

So the way I understand it, rescheduler add-on uses the global ones (above), not deck-specific ones (below)?

    } else if (deck_name.startsWith("Main::Japanese")) {

        var f_s = [0.3047,1.0348];
        var f_d = [1.0066,-0.365,-1.7372,0.2008];
        var s_w = [3.3204,-1.4175,-0.1381,1.5241,1.6589,-0.5137,0.8736,1.2989];
        // User's custom parameters for sub-decks

[Expertium]

Maybe what you need is to monitor the requestRetention and maximumInterval in the custom scheduling code, and automatically update the intervals of cards when you modify these two parameters. But it requires the add-on to parse the code and react to the modification in live. I think Anki doesn't support it by far.

I don't think it needs to monitor them "live" (though that would be ideal), but as long as the add-on can parse code when I click "Reschedule cards" I'd say it works.

[L-M-Sherlock]

So the way I understand it, rescheduler add-on uses the global ones (above), not deck-specific ones (below)?

There are some misunderstandings. Rescheduling doesn’t use the parameters. It uses the stability of card. The stability is calculated when you review the card. And the calculation used the deck-specific parameters if you set the deck name in scheduler’s code.

[L-M-Sherlock]

I don't think it needs to monitor them "live" (though that would be ideal), but as long as the add-on can parse code when I click "Reschedule cards" I'd say it works.

It seems feasible. I will try in next day. I am going to bed now. It is 3:48 AM in China. See you tomorrow.

[Expertium]

And the calculation used the deck-specific parameters if you set the deck name in scheduler’s code.

So it can parse the code from the window in deck options, right? Well then, I will be eagerly waiting until you automate it. Good night man. And add a link to github to the add-on page on ankiweb, otherwise people will be confused.

[L-M-Sherlock]

And you can see this page: https://github.com/open-spaced-repetition/fsrs4anki/wiki/Auto-migrate-from-SM-2-to-FSRS It didn't use the parameters to calculate the stability of existing cards.

[Expertium]

I guess we were talking about different things. Let me explain precisely what I meant by "recalculating intervals":

1. Parse the entire review history of a card
2. Calculate S(D,S,R) for each review using either global or deck-specific parameters
3. After the last review calculate the optimal interval according to FSRS
4. Replace the current interval (given by Anki) with the interval calculated above Suppose a card has been reviewed 10 times. After the first review D, S and all other stuff is calculated using global or deck-specific parameters. Then the same procedure is repeated after the 2nd review, then after the 3rd, then after the 4th, etc. Then, once S for the last 10th review is calculated, use it to calculate optimal interval. Suppose after 10 reviews the interval given by Anki is 1 year. You replace that value with whatever value FSRS produces. It's pretty much the same procedure you do when parsing review history for the optimizer and calculating S(S,D,R) for the optimizer.

[L-M-Sherlock]

It is a big project. I figured out how to get the code of custom scheduling in the Anki add-on today. And Anki doesn't supply the direct method to get a card's entire review history. I need to implement it from scratch. Besides, I can't use some powerful python libraries to simplify the work because Anki doesn't include them (For details, please see: https://addon-docs.ankiweb.net/python-modules.html).

[Expertium]

And Anki doesn't supply the direct method to get a card's entire review history.

I thought the entire review history of a card is necessary to calculate the new interval when I press Again/Hard/Good/Easy? I mean, values of S and D depend on their own previous values, so to calculate the current values of D and S you need the entire review history, no? Or is it different if you want to do that for all cards and not just one?

[L-M-Sherlock]

I thought the entire review history of a card is used necessary to calculate the new interval when I press Again/Hard/Good/Easy?

If the card includes the stability and difficulty calculated in the last review, the entire review history is unnecessary.

[L-M-Sherlock]

So the current method of processing the existing cards is a simplified version. It assumes the interval given by the built-in schedule is an approximate value of stability.

[Expertium]

So the current method of processing the existing cards is a simplified version. It assumes the interval given by the built-in schedule is an approximate value of stability.

Ok, thank you, now I understand. Well, hopefully you'll implement the full version one day.

[L-M-Sherlock]

I update the add-on https://ankiweb.net/shared/info/759844606

And now, it can reschedule all cards based on parameters stored in the custom scheduling code.