Optimiser Stops Working After a Single Review

[brishtibheja]

Hey there, I am facing an issue which I've already encountered twice.

I tried optimising my parameters for one of my decks which works fine. I don't save the settings, go back, do a review card, and now the optimiser shows FSRS parameters are already optimal. Here's the recording of what I was doing: Youtube

[brishtibheja]

AnkiDroid Version = 2.18beta1 (c12b398c216dc2a2c2e2c98709c92132d19d105c)

Backend Version = 0.1.38-anki24.04.1 (24.04.1 ccd9ca1a8309b80bcb50ddc5d99c7ce63440bce9)

Android Version = 11 (SDK 30)

[brishtibheja]

More context: The parameters look something like this,

0.3146, 0.6462, 1.9019, 3.9061, 5.2106, 1.2440, 0.8834, 0.0355, 1.4877, 0.5973, 0.8750, 2.4010, 0.0164, 0.5967, 1.6725, 0.5179, 2.6010

After optimisation they become,

0.3854, 0.7023, 1.7281, 3.1494, 5.2575, 1.3254, 0.9625, 0.0188, 1.5783, 0.5888, 0.9660, 2.3874, 0.0208, 0.5538, 1.6276, 0.5041, 2.5553

[brishtibheja]

Also here's the deck if this helps. I've not included the media or anything else but it comes with the same scheduling.

漢字 Writing-20240508151559.zip

Note: This is NOT a zip! I merely renamed the file so as github allows it to go through.

[L-M-Sherlock]

Please report it in https://forums.ankiweb.net/ or https://github.com/ankidroid/Anki-Android/issues

I guess the client cached the previous optimized parameters even when you didn't save them. But I don't know how to reproduce it.

[brishtibheja]

But I don't know how to reproduce it.

You can set the date of your device to "8th May" i.e., yesterday, and import my deck and repeat the steps. I will report it on the AnkiDroid issue tracker.

[L-M-Sherlock]

I don't have AnDroid device. Could you reproduce this bug in Anki PC?

[brishtibheja]

... And I don't have a PC 🤦

Actually, now that I think about it, the issue might not be AnkiDroid caching the previously optimised parameters. I say this because it doesn't happen always. I have previously optimised (without saving) and then did a relearning card, after which I could optimise just fine. Then I do a review card and now the optimiser isn't working.

[L-M-Sherlock]

I tried to reproduce this issue in my PC but failed. Does this issue happen frequently?

[brishtibheja]

Well if I had to choose between frequently or not frequently, I'd say the former. I've a habit of leaving the review screen mid-review and wandering around in the app, sometimes deck options. That's how I found out that sometimes in some seemingly random times the optimiser can improve the parameters but when you do your review cards and leave that "junction" it stops working.

I tried to reproduce this issue in my PC but failed.

If you mean you tried with my deck then it's possible. When I tried re-producing this on my Android, applying the deck options to all the subdecks would destroy the order of the review cards with which I exported the deck. From my personal experiences I believe the order of review cards is important here.

[L-M-Sherlock]

sometimes in some seemingly random times the optimiser can improve the parameters

The optimizer of FSRS is based on stochastic gradient descent method. So it's possible that it cannot find out a better set of parameters. And it's unrecommended to optimizing your parameters frequently. Once a month is enough.

[brishtibheja]

And it's unrecommended to optimizing your parameters frequently. Once a month is enough.

Yes I'm completly aware. But still, I brought my RMSE down to 5.67 from 6.12 so a plus for me.

[L-M-Sherlock]

What's your log loss? Due to technical limit, the optimizer only uses log loss to optimize the parameters.

[brishtibheja]

I didn't see your message. Apologies for the late reply. I would never want to keep anyone waiting so don't hesitate to ping me if you ever want to.

What's your log loss?

Depends on "when" as it has changed a lot. Keep reading. When I first opened this issue, and I checked the screen recording here, log loss was .4248 which decreased to .4240 after the optimisation.

I encountered the same issue once again the next day (or is it opportunity?) which got the log loss down to .4238 (I'm 98% sure about this; I found some scribbles on my kanji writing notebook)

It seems to me that my RMSE/log loss also changes on it's own. 5 days ago RMSE was 5.67 as I told you previously. Now it's 5.57 and I'm assuming this is expected. My log loss on the other hand increased to .4246 which is still lower than the initial figure of .4248 log loss.

Another point, and this is a bit off-the-tangent. I hear about improving the D and just the scheduler in general and bringing average-RMSE down (mainly from Expertium) but not enough about the optimiser. I believe the needful might be to just improve the optimiser such as people get more personalised parameters. This issue, also, might just show that optimiser is sensitive to when are you trying to use it?

[L-M-Sherlock]

I hear about improving the D and just the scheduler in general and bringing average-RMSE down (mainly from Expertium) but not enough about the optimiser.

These discussions are about the optimizer or the model. I guess you mean the hyper-parameters?

This issue, also, might just show that optimiser is sensitive to when are you trying to use it

It has been researched:

If you re-optimize your collection every 1000 reviews, you will have a 76% chance of getting better parameters.

[brishtibheja]

These discussions are about the optimizer or the model

I might be hearing more of the later then. Never mind.

It has been researched:

Oh great! So it's already something known. Personally it was surprising for me to see just an additional review making the optimiser unable to find better parameters.

[Expertium]

These discussions are about the optimizer or the model. I guess you mean the hyper-parameters?

He meant the optimizer itself, the stuff that adjusts parameters and finds the optimum.

Oh great! So it's already something known. Personally it was surprising for me to see just an additional review making the optimiser unable to find better parameters.

(I'm assuming it was you who commented on Reddit recently and mentioned the optimizer). The optimizer itself was improved back when FSRS v3 was the latest version. There was a problem with the optimizer not selecting the best parameters, and instead selecting the parameters from the last training epoch, which aren't always the best. We fixed it. Then I believe there was some other problem that LMSherlock noticed and fixed, but I don't remember what it was. At this point there aren't many ways to improve the optimizer. We could increase the number of training epochs, but it would make the optimizer slower, and anything after 5 epochs barely makes a difference anyway, so the improvement would be tiny. We could maybe try different batch sizes? @L-M-Sherlock I suggest trying several different batch sizes and reporting speed and log-loss/RMSE.

[L-M-Sherlock]

Small batch size means more iteration and more stochastic optimization in each step.

According to the research "An Empirical Model of Large-Batch Training " from OpenAI:

But their dataset is pretty large. I think our case is very different from theirs.

[brishtibheja]

Hey, I was able to reproduce the issue today. If you're interested you can try with this deck. It has only one review card so there shouldn't be a problem with ordering of review cards.

漢字 Writing-20240522012527.zip

Once again, rename the extension to .apkg and import it. The date should be set to yesterday's, i.e., 21st May.

[L-M-Sherlock]

emmm, Anki doesn't allow me to run it in an unsynchronized date:

[L-M-Sherlock]

I'm afraid that I cannot solve this issue because it's the inherent to stochastic gradient descent. The new review will change the batching results of items, and it may induce a butterfly effect during the optimization process.

[brishtibheja]

Oh that is sad. I hope in the future we have better methods for optimisation. I'm optimistic. There is nothing concrete to do here so should we close this issue then?

[L-M-Sherlock]

Increasing the number of epochs and decreasing the learning rate could make the optimization more stable, but it will slow down the optimization and it cannot solve it completely.

Another obstacle is the different metrics used in comparing parameters and optimizing them. We use RMSE(bins) to evaluate parameters, but log loss is used to optimize them.

[brishtibheja]

Hey @L-M-Sherlock I understand this is irrelevant now, but may come in help in the future. I realised you could just change the "Next day starts at" setting instead of trying to change the actual time. You can even set a 23 hours delay for the starting of the next day! Anyways, thanks for your assistance here.