[Bug] Scheduler is too slow

osavoskin commented 3 years ago

Reproduction Steps

Sometimes it takes up to a few seconds to get you to the next card. The DB is not that large and optimized. It started after the latest update.

Expected Result

Scheduling should be almost immediate

Actual Result

Takes a few seconds

Debug info

AnkiDroid Version = 2.15.4

Android Version = 11

Manufacturer = samsung

Model = SM-G973F

Hardware = exynos9820

Webview User Agent = Mozilla/5.0 (Linux; Android 11; SM-G973F Build/RP1A.200720.012; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/91.0.4472.101 Mobile Safari/537.36

ACRA UUID = 350da5e9-8882-4756-b191-9b787906a920

Scheduler = std

Crash Reports Enabled = true

DatabaseV2 Enabled = true

mikehardy commented 3 years ago

This looks like the exact sort of thing @Arthur-Milchior might be interested in ;-) @osavoskin are able to share your collection? Timing issues like this are data-dependent, and we likely won't be able to conclusively figure out the problem unless we are working on the same collection that shows the problem. If you can, just zip collection.anki2 from /sdcard/AnkiDroid and attach it here (github will take zips) If you can but it is slightly sensitive then you might be able to share just a controlled link directly with Arthur (if he's interested and has time to take a look) or with whomever does have time

osavoskin commented 3 years ago

Thank you for such a quick reply! The database you asked

Arthur-Milchior commented 3 years ago

You didn't provide access to it. When you mention moving to the next card, do you mean on each review? Only after some special operation such as undo/bury/suspend? on first card only?

osavoskin commented 3 years ago

Fixed the access. Yes, it happened multiple times during regular review. Unfortunately I didn't notice any patterns that might precede the freeze. In terms of installed software the phone is clean, almost nothing extra. It definitely cannot be related to insufficient computing resources

Arthur-Milchior commented 3 years ago

For futur bug, I'd appreciate you export from anki tool instead of just copying the .anki2 file, because I could not easily import it back. I've dealt with it, don't worry, just for a next time

When you say "slow" how long do you mean? one second? 10?

Arthur-Milchior commented 3 years ago

I just put your collection on my device. A SM-202F. I saw nothing unexpected. Admittedly, it is not instantaneous and that is a shame, but there is nothing new here. I used the profiler and so nothing in the stack that was not there in .14, apart from the call to rust database. But the whole getCard function returns in .08 seconds, with the profiler turned on. Even the optimization I've been arguing for, for a long time, of precomputing next card content would really not help here.

I can imagine that for some reason, rust backend is slower on some device, but I'd need to run the profiler on the actual device to check it.

However, there is something strange, which is that this collection has two tables sqlite_stat1 and sqlite_stat2 that is new to me. They also have a lot of filtered deck, and the app crashes when I try to open then. However, since it's not a standard way to import a collection it may be related to the way export was done.

Did you take the file from ankidroid folder or from an anki desktop?

osavoskin commented 3 years ago

This is the exported collection.

Exported from the mobile app. Please, see the debug info for env. details.

The freezes were max 2sec. long. I realize it is not that critical, but it never happened before. I would say I have never seen a freeze on this phone, that's why I filled the bug.

I will reinstall the app and reimport the collection. Will see if it helps

Arthur-Milchior commented 3 years ago

Even if not critical, I understand very well it might be a pain. I spent I don't know how many hours trying to save every second possible, it should really not take 2 seconds for such a simple operation.

But even if I agree with you, if I can't reproduce, I don't know what I can do about it. You have a very simple collection, so it is not even the complex case I have dealt with in the past.

For the record, there is generally no DB access to fetch the next card. The one or two potential next cards are precomputed while you review a card. There is still two database access to save the review (in log and updating card). In theory, we could show the next card and then save the card, that would save a tenth of second maybe. But it should not be relevant to your problem. Might be worth opening an issue however. It's not entirely trivial in the case where the current queue of cards to review is empty.

While I doubt it's DB related, if I would love to test with former DB if possible, just to double check if there is an option to use it back. Might be worth discussing as long as the former code is not deleted. Do you think it could be possible @david-allison-1 to have a way to let user test the former DB just to check whether there is a speed difference for them?

david-allison commented 3 years ago

While I doubt it's DB related, if I would love to test with former DB if possible, just to double check if there is an option to use it back. Might be worth discussing as long as the former code is not deleted. Do you think it could be possible @david-allison-1 to have a way to let user test the former DB just to check whether there is a speed difference for them?

Very easy. See #9066 for where the logic is to select the Java backend.

I wouldn't oppose it in dev mode, but this will clash with #9066, so only temporarily. Using the new Anki backend locks the database, so it won't be possible going forwards.

I wouldn't rule out the possibility of it being DB related. If it is then there's a pending issue in the backend which should speed up small queries. This would be my second suggestion, after putting in a performance regression test.

Arthur-Milchior commented 3 years ago

I will reinstall the app and reimport the collection. Will see if it helps

Dit it help? Would you mind filming your reviewing for a minute or two? Maybe if I can see it, I could detect a pattern that you would miss, that would give hints about what cases are slow? Please, if you capture the video, think about indicating where you click, so that we can easily see everything you do. See e.g. https://android.gadgethacks.com/how-to/show-screen-touches-with-samsungs-new-built-screen-recorder-0233105/

osavoskin commented 3 years ago

Sorry for the delay in response. I think we can close the issue, as I wasn’t able to reproduce it after reinstalling the app and doing resync with the server. Sorry for the false alarm

Arthur-Milchior commented 3 years ago

I am going to insist vehemently, please, dont not present excuse for giving feedback. I've never heard the problem of "we got too much detailled feedback with discussion from a user". I extremely strongly to prefer a few false positive costing an hour or so than to learn that a user had to reinstall the app every single day and didn't even thought about contacting us to make a bug report (that is, sadly, a true story. We learned about it later, when the bug was corrected)

osavoskin commented 3 years ago

Thank you for your time! I will definitely let you know if I have any meaningful feedback or encounter a new issue

oakkitten commented 2 years ago

I have this issue on Xiaomi Mi A2 (Android 11) on current main, it's intermittent. Here's when it's good (wall clock):

And here's when it's bad:

Well, this is not very helpful ¯\_(ツ)_/¯

mikehardy commented 2 years ago

Really long single queries...really long text processing. Is the card template huge or similar? Is the collection or deck very large? There is some pre-fetch at work I think so intermittency might be that sometimes you are reviewing things that are pre-fetched and sometimes you get to a card before it's fetched. I think if you watch the adb logcat you can see the queries that are going in, and you can maybe determine at a lower level what that query is returning (maybe by opening the sqlite db in a desktop sqlite editor

Also you might be able to instrument that code and see what's going on during the loooong green horizontal stretches

https://github.com/ankidroid/Anki-Android-Backend/blob/0c095aa8eb6f6b72bd88661d44fdf9a630f34f45/rsdroid/src/main/java/net/ankiweb/rsdroid/NativeMethods.java#L70

goes here https://github.com/ankidroid/Anki-Android-Backend/blob/0c095aa8eb6f6b72bd88661d44fdf9a630f34f45/rslib-bridge/src/lib.rs#L385

https://github.com/ankidroid/Anki-Android-Backend/blob/0c095aa8eb6f6b72bd88661d44fdf9a630f34f45/rsdroid/src/main/java/net/ankiweb/rsdroid/NativeMethods.java#L91

goes here https://github.com/ankidroid/Anki-Android-Backend/blob/0c095aa8eb6f6b72bd88661d44fdf9a630f34f45/rslib-bridge/src/lib.rs#L450

Real fishing expeditions there but if you want to play around, there's some handles on it :fishing_pole_and_fish:

oakkitten commented 2 years ago

The card template is nothing out of ordinary, a bit of CSS and 5 fields. Some 16k/20k cards in the deck/collection, and around 500k reviews in total. By intermittent I meant, I can review 50-100 cards with a noticeable delay, and then it goes away and I can review 50-100 cards without it. It doesn't vary from card to card.

Instrumenting rust sounds like quest I'm not yet prepared to brave Maybe one day. One fine day, one that is not today.

mikehardy commented 2 years ago

Instrumenting rust sounds like quest I'm not yet prepared to brave image Maybe one day. One fine day, one that is not today.

with you on that one

I wonder if there is something else running on your phone that is causing a huge draw on CPU and/or storage hardware available throughput?

oakkitten commented 2 years ago

I've had this issue for a while, I think if something was occupying the CPU I would notice. Speaking of disk, google says that some users report issues with the storage of my particular Xiaomi device. I personally never experienced any issues with storage, and I've had this device for years, but perhaps they manifest themselves with AnkiDroid somehow?

ankidroid / Anki-Android