Support next review date in IR

tvhong commented 2 years ago

Is your feature request related to a problem? Please describe. Currently, the Priority Queue feature in IR add-on only supports a priority value for each card.

There are two main ways a user can use the priority queue:

Assign priority number for each card and frequently "randomize" the priority queue.
Does not use per-card priority number and use the priority queue as a first-in-first-out queue.

However, both these approaches have shortcomings when a user has many cards in the queue (hundreds). Approach (1) prevents the user from ever getting down to the lower priority cards. Approach (2) means the user spends too little time on the important cards and too much time on unimportant ones.

In SuperMemo, apart from a priority, each topic (equivalent to IR card) also has a next review date. Everyday, SuperMemo creates a list of outstanding repetitions for all topics whose "next review date" is on that day. Additionally, topics within the outstanding repetitions queue are ranked based on their priority. During the learning session, the user reviews the highest priority topic first. And if the user does not finish the outstanding repetition queue for that day, subsequent topics are rescheduled for a later day.

The algorithm for rescheduling leftover topics from the outstanding repetitions queue is described below:

the algorithm for determining inter-review intervals for topics is much simpler and is entirely under your control. Each article receives a specific priority. The priority determines which articles are reviewed first and which can be postponed in case you run out of time. Each article is also assigned a number called the A-Factor that determines how much intervals increase between subsequent reviews. For example, if A-Factor is 2, review intervals will double with each review. Priority and A-Factors are set automatically, but you can change them manually at any time. Priorities and A-Factors are determined and modified heuristically on the basis of the length of the text, the way it is processed, the way it is postponed or advanced, and by many other factors.

http://super-memory.com/help/il.htm

Describe the solution you'd like

Each IR card has "next review day".
Support daily "outstanding repetitions" queue where IR cards are ranked based on priority.
Users can view and edit next review day for IR cards.
Untouched IR cards from "outstanding repetitions" are re-scheduled for a later date.

tvhong commented 2 years ago

Here are some useful resources related to this:

SuperMemo wiki youtube: Starting with Incremental Reading by going through the IR Manual
This add-on's (Incremental Reading) implementation of priority queue: https://github.com/tvhong/incremental-reading/blob/master/ir/schedule.py#L210
Searching, PDF Reading & Note-Taking in Add Dialog implementation of priority queue: https://github.com/fonol/anki-search-inside-add-card/blob/9688243645c5f3ac7bfe13dd8e76a901e45cd641/src/notes.py#L721
SuperMemo wiki on Incremental reading for users of spaced repetitions: https://supermemopedia.com/index.php?title=ABC_of_incremental_reading_for_any_user_of_spaced_repetition

tvhong commented 2 years ago

The algorithm for priority queue in SIAC addon (aka Searching, PDF Reading & Note-Taking in Add Dialog) is interesting.

It also supports a single queue for all articles. Simplistically, the algorithm for determine ranking within the queue can be defined as:

# priority between 1-100 with 100 being the highest
# higher rank means higher position in queue 
rank = priority * timeSinceLastReview

This means that between topics with same "timeSinceLastReview" value, ones with higher priority will appear earlier in the queue. And between topics with same priority, ones with longer timeSinceLastReview will appear earlier.

This seems like a fairly good algorithm and clearly has served the author of SIAC very well.

However, in order to correctly compare topics with different priority and timeSinceLastReview, we'll need a bit more calibration. The author of SIAC did this by introducing PRIORITY_SCALE_FACTOR and PRIORITY_MOD configurations. Additionally, they needed special logic for days_delta < 0.5.

# How many times more important is a priority 100 item than a priority 1 item?
PRIORITY_SCALE_FACTOR   : int           = get_config_value_or_default("notes.queue.priorityScaleFactor", 5)

# how should priority be weighted
PRIORITY_MOD            : float         = get_config_value_or_default("notes.queue.priorityMod", 1.0)

def _calc_score(priority: int, days_delta: float) -> float:
    prio_score = 1 + ((priority - 1) / 99) * (PRIORITY_SCALE_FACTOR - 1)
    if days_delta < 0.5:
        return days_delta + prio_score / (PRIORITY_MOD * 10000)
    else:
        return PRIORITY_SCALE_FACTOR * days_delta + PRIORITY_MOD * prio_score

https://github.com/fonol/anki-search-inside-add-card/blob/9688243645c5f3ac7bfe13dd8e76a901e45cd641/src/notes.py#L721-L726

I spent sometime thinking about how we could simplify the algorithm but came up with the exact same 2 configuration values that SIAC introduced. However, from a normal user's point of view, the exact meaning of these 2 configurations are quite mysterious.

At this point, I think it is more difficult to understand the SIAC-like algorithm than the SuperMemo's priority queue algorithm.

tvhong commented 2 years ago

So, here's how I envision how next review date in IR would look like:

Each IR card has a priority between 1-10 with 10 being the highest (same behavior as before)
Introduce an "outstanding queue". This queue contains cards to be reviewed each day, sorted by priority.
Convert the organizer view to be a normal table that can sort by card names or other columns.
Add a "next review date" column to the organizer view.
Allow users to edit next review day in the organizer.
After each review, the user can change the card's priority or just press spacebar to finish the review.
Un-reviewed cards will be automatically postpone for a future date.
Cards that are reviewed or postponed have their next review date increased from the last time. How much this increases depends on the card's priority.

Here's a rough outline of the scheduling algorithm:

# priority 1-10 with 10 being highest
newInterval = prevInterval * (1 + 1/priority)

So, for a priority 10 item with prevInterval = 5 days, newInterval = 5 days * (1 + 1 / 10) = 5.5 days. And for a priority 5 item with prevInterval = 5 days, newInterval = 5 days * (1 + 1 / 5) = 6 days.

And to rank items in the Outstanding Queue, we'll use (priority, lastReviewTime). Items with higher priority (10) will be higher in the queue. If they have the same priority, the one with larger lastReviewTime will be higher. Using lastReviewTime this way, we get the benefits that extracts from the same article will show up in order, and before the main article.

tvhong commented 2 years ago

Describing the above from a user's perspective:

Scenario 1: adding new articles

(This is similar to the current experience) Import an article and specify the priority of the card. The card's next review date depends on its priority.

Scenario 2: daily review

Open an IR deck. Read the first article or extract from the Outstanding Queue. After finish reading, press "Show Answer". In the "back" of the card, modify the priority if desired. Once done, click "Next". The reviewed card will be scheduled for a future day. Repeat until all cards in Outstanding Queue is reviewed. If there are remaining cards by the end of the day, they are auto-postponed to a future day.

Scenario 3: reviewing outstanding queue

Open the outstanding queue via Read > Outstanding. See a modal with articles and extracts in priority order. This modal is read-only. To edit, switch to the Organizer view.

Scenario 4: changing next review date or priority

Open Organizer view from Read > Organizer. Select some cards and click "Change next review date". Enter new value and save. If the next review date is today, the card is added to the Outstanding Queue.

Scenario 5: reset interval (not sure if this needs to be an explicit feature yet)

Open Organizer view from Read > Organizer. Select some cards and click "Reset interval". This will reset the next review date to 1 and restart the interval calculation

tvhong commented 2 years ago

We can store additional fields into cards from Anki 2.1.55. This might be useful. https://github.com/ankitects/anki/issues/2039

tvhong commented 2 years ago

I'm implementing this idea in https://github.com/tvhong/incremental-reading/tree/vhong/pq2 . I can get the Outstanding queue and the scheduling working. But cannot show the cards according to the order in the Outstanding queue.

I tried overriding Reviewer._get_next_v1_v2_card and Reviewer._get_next_v3_card in that branch (reference), but that doesn't work because Reviewer keeps track of cards in its internal data structure.

I could probably override getCard method in the v2 scheduler. But the v3 scheduler uses get_queued_cards, which queries data directly from rust backend, is a lot harder to override.

tvhong commented 2 years ago

Checking other addons that change review orders, they only support scheduler v2.

Examples:

https://ankiweb.net/shared/info/3731265543

tvhong commented 2 years ago

I'm gonna put this one on hold.

Don't wanna add a new feature that does not work on the latest scheduler.

tvhong commented 2 years ago

Here's a post to ask about how to override review order on Anki's forum: https://forums.ankiweb.net/t/how-to-override-review-order-in-scheduler-v3/23786

tvhong / incremental-reading