Web UI Lacks Feedback Sometimes

[beville]

This is a bit more nebulous to describe, but this is an issue I have when browsing with the web interface. Often when changing views or searching, Codex is slow to respond (often 5, 10, or more of seconds), but there is no feedback on the screen.

This can happen if I change the "top group" or "order by" drop downs, hit the "Up to" buttons, or do a search. Basically, nothing happens immediately, and I have no feedback to tell me to be patient. I might end up changing the above again before the operation is done, which seems to queue up more changes, which can also be slow.

Obviously this is highlighting some performance problems, but that's a separate issue, I think.

[ajslater]

Yeah. This opens up two questions.

Why does it pause like that? I really don't know. Some big complicated queries are likely to take milliseconds longer, but not 10 seconds. Is it waiting for a disk to spin up?

When this happens, what should the the UI do? Currently when the browser makes a request to filter stuff it sets a bit that says "show that big turning placeholder circle" and then it shuts it off when it finishes processing the response and its built the proper browser cards for display. But I guess that might not happen sometimes and also that big circle is kind of ambigious.

[bmfrosty]

Any quick and easy way to turn on some sort of slow query tracking for the database? I suspect a very large library could do this - I just don't know a way to track it.

On Tue, Mar 26, 2024 at 2:43 PM AJ Slater @.***> wrote:

Yeah. This opens up two questions.

Why does it pause like that? I really don't know. Some big complicated queries are likely to take milliseconds longer, but not 10 seconds. Is it waiting for a disk to spin up?

When this happens, what should the the UI do? Currently when the browser makes a request to filter stuff it sets a bit that says "show that big turning placeholder circle" and then it shuts it off when it finishes processing the response and its built the proper browser cards for display. But I guess that might not happen sometimes and also that big circle is kind of ambigious.

— Reply to this email directly, view it on GitHub https://github.com/ajslater/codex/issues/357#issuecomment-2021525090, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAHYDX3PKL5CIA4XBY3Z53Y2HTZBAVCNFSM6AAAAABFJJKPH6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRRGUZDKMBZGA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[beville]

Why does it pause like that? I really don't know. Some big complicated queries are likely to take milliseconds longer, but not 10 seconds. Is it waiting for a disk to spin up?

Pretty sure it's not the disk spinning up.

So I do have have Codex running on a not very beefy Celeron 2GHz/8GB RAM machine with other media servers running on it, but it's able to do most of this stuff quite well.

I just installed Codex on a faster NUC Core i7/16GB desktop here, and imported the slimmed down version of my library. The import ran about 4 time faster on the i7, which is what I expected.

i7:

2024-03-27 02:56:16 UTC INFO    Updated library /comics in 9 minutes. Imported 17876 comics at 30.2 comics per second.

celeron:

2024-03-24 18:10:39 UTC INFO    Updated library /fake_comics1/slimlib in 41 minutes. Imported 17876 comics at 7.1 comics per second.

I'd be curious if you tried on your own machine to import that same library that I shared, and then run this test:

# Not sure if this is needed, but want to possibly clear any RAM caching
docker restart codex
# wait for restart to complete
time wget -O - "localhost:9810/opds/v1.2/s/0/1?q=batman" >results.txt

Here are the times I saw for that query (which returned 100 results)

celeron    2m29.342s
i7         1m05s

Starting fresh and running that same query in the Web UI saw similar times before the results showed.

Even more interesting is that after query, trying to hit the "up to" buttons (or maybe other stuff) Codex seems to go off the rails, spawning a few busy threads, and not much happening on the UI.

I think I was seeing this a lot of the time before you fixed the "failed search returns all issues", which is why I've had a tough time with the Web UI? Not sure. I'm sure that fix will help reduce inadvertent query results, though. (Thanks for fixing that!)

When this happens, what should the the UI do? Currently when the browser makes a request to filter stuff it sets a bit that says "show that big turning placeholder circle" and then it shuts it off when it finishes processing the response and its built the proper browser cards for display. But I guess that might not happen sometimes and also that big circle is kind of ambigious.

I don't think I ever see the big circle when changing "top group" or "order by", or when doing a search. But something like that would probably be better than nothing? I'm not sure!

I've been wondering, if type in a query, hit enter, and then while waiting for the result, if I hit enter again, does that spawn a new query thread, either in parallel, or queue up a new query, or is it ignored?

Ideally a "Your query is in progress" status would appear, with an option to cancel it, maybe. I have no idea if that's viable, or maybe wouldn't even be needed if the root cause of the delays was identified.

Anyway, I'm curious what your experience is with that test library I shared. If you do have similar problems, maybe that will help track down what I'm seeing. If not.. dunno.

[abillauer]

Not sure if this helps but I see similar behavior on 1.5.9. With a large library going up a level in folder view can take a VERY long time with no feedback from the screen. Happy to do any testing needed to help isolate the issue.

[abillauer]

I did some further experimentation for this. This issue is strongly felt on older/slower systems. Using the same library on both my RPI 4 and M1 Mac:

On my RPI4 this is visible where you can click on a folder and wait 15-25 seconds. On my M1 Mac the issue is not present/noticeable.

Not scientific but the issue is much more pronounced on weaker processors.

[ajslater]

Yeah that's interesting. My M1 Max with SSD dev machine does that slimlib with: 4 minutes. Imported 17876 comics at 60.1 comics per second.

Might be worth my reexamining query complexity.

[beville]

Yeah, it does feel like a query optimization problem, which I know is probably not trivial.

Did you try that query with "batman" on that library? Based on import time, your Mac is about twice as fast as my desktop, and 10x my media server. I think query times will scale differently, though.

[abillauer]

To help diagnose the issue can the debug log level record UI clicks, the query that is run, and response time?

[ajslater]

I'm experimenting with query loggers analyzers and timers right now, and while the db query times seem ok, i am seeing some suspicious delays elsewhere in the browser code even on a fast machine. Gonna look into this further and hopefully produce some user activatable diagnostic tools like you suggest as well so even if i solve this issue we can get data and early warnings in other cases.

[beville]

The additional appearances of loading icon in the latest version makes big difference when navigating. Even if the results aren't any quicker, having an instant response feels a lot better. Thanks!

[abillauer]

Agreed. Not sure if it is faster but it definitely feels faster!

[abillauer]

The web performance is much improved in v1.5.14!

Thanks so much for you hard work on this!

[ajslater]

I'm gonna close this issue because it was opened about UI feedback and I think that's finished.

But again i must thank you for bringing this up as it spurred me to take a closer look at search and there are some speedups there coming in the near future.