search

vit-project / vit

VIT is a lightweight, fast, curses-based front end to Taskwarrior
MIT License
492 stars 51 forks source link

second(s) of delay for most operations with largish db #358

Closed imyxh closed 3 months ago

imyxh commented 3 months ago

With my taskchampion.sqlite3 of 1004 KiB, it takes about a second even just for Add: to appear in the bottom of the screen after typing a. Most other operations (starting vit, syncing, etc.) incur similar or worse delays. Running e.g. task add test either through vit or on its own similarly takes another second or so. If I move my taskchampion database somewhere else, everything is of course works very fast.

This occurs in alacritty, tmux, and the vterm. Any suggestions to narrow this down? I feel like ~1 MiB is really not that big of a database, even for my Core 2 Duo.

scottkosty commented 3 months ago

Do you think this is a vit issue or a taskwarrior issue? From what I understand, if you see the same delay with "task add test" then it is a Taskwarrior issue so it might be good to check over there.

thehunmonkgroup commented 3 months ago

There have been reports of performance issues with TaskWarrior 3.x. If you're running task from CLI and getting these delays, this is a TaskWarrior issue, not a VIT issue.

Closing. If you can provide specific evidence that the performance issue is coming from VIT, please file a new issue with the details.

imyxh commented 3 months ago

Do you think this is a vit issue or a taskwarrior issue? From what I understand, if you see the same delay with "task add test" then it is a Taskwarrior issue so it might be good to check over there.

To be clear, it takes about one second for VIT to show me the the Add: prompt, and then once I type test\n it takes about another second to run that. With Taskwarrior's CLI I only have the latter delay. So two seconds with VIT, one second with Taskwarrior.

It's indeed true that Taskwarrior should be faster but there's also no reason why VIT should have to take so long to show me the Add: prompt, no?

scottkosty commented 3 months ago

Thanks for those details! From what I understand, VIT takes a second to start up because it has to run "task" in the background to read your tasks. So again in this case, although it seems like VIT is taking the time, it's really the underlying call to Taskwarrior that's taking the time.

imyxh commented 3 months ago

Thanks for those details! From what I understand, VIT takes a second to start up because it has to run "task" in the background to read your tasks. So again in this case, although it seems like VIT is taking the time, it's really the underlying call to Taskwarrior that's taking the time.

Calling task on startup to see the list of tasks makes sense. But does task need to be called (e.g.) in the time between when I hit the a key and when the Add: prompt appears?

Basically I suspect vit is calling task more than it needs to.

scottkosty commented 3 months ago

Calling task on startup to see the list of tasks makes sense. But does task need to be called (e.g.) in the time between when I hit the a key and when the Add: prompt appears?

Basically I suspect vit is calling task more than it needs to.

ahhh now I see. I agree with you. I'm not sure why VIT needs to call task after you hit the 'a'. @thehunmonkgroup do you know if this is indeed expected?

thehunmonkgroup commented 3 months ago

Basically I suspect vit is calling task more than it needs to.

VIT calls task when needed, and not otherwise.

In this case, it must call task to build the autocomplete settings for the add functionality.

The performance of the task binary is not something this project has control over -- those issues should be reported to the Taskwarrior team.

You can also consider rolling back to the 2.6.x series of TaskWarrior. I personally will not upgrade to 3.x until several major issues with the new release (including this one) are resolved.

scottkosty commented 3 months ago

In this case, it must call task to build the autocomplete settings for the add functionality.

Ah that make sense. Could it improve things to build the autocomplete settings only when Tab is pressed?

Additionally, would it make sense for the autocomplete settings to be cached (e.g., for subsequent Adds)? It might be a bit of a pain to deal with when the cache should be invalidated (e.g., a new tag has been introduced), but seems feasible. I'm not suggesting you work on this, I'm just wondering if it would make sense.

thehunmonkgroup commented 3 months ago

Ah that make sense. Could it improve things to build the autocomplete settings only when Tab is pressed?

I think this would be a pretty bad user experience.

Additionally, would it make sense for the autocomplete settings to be cached

I can only imagine the number of bugs that would happen because of how hard it can be to manage a cache.

IMO, we should not be refactoring VIT because TaskWarrior has performance issues -- these issues should be handled where the problem is occurring -- in TaskWarrior.

Traditionally, with the exception of giant pending.data files, task commands have been lightning fast. IMO, if there's a 1 second delay for every single task command, then TaskWarrior will be unusable, and no amount of adjustments in VIT will change that.

scottkosty commented 3 months ago

Ah that make sense. Could it improve things to build the autocomplete settings only when Tab is pressed?

I think this would be a pretty bad user experience.

In the case of lightning fast "task", it wouldn't make a difference either way. In the case with a slower "task", I'm not sure if it would be more confusing to have it on generating the "Add" prompt or on tab. In some sense, the slowdown on "Tab" would be easier to understand because then it would have been obvious to me that it has to do with the tab completion, where as I didn't guess that for the slow down on "Add" so that to me was more confusing. Further, I personally don't use tab completion so I wouldn't be hit by that performance issue.

IMO, we should not be refactoring VIT because TaskWarrior has performance issues -- these issues should be handled where the problem is occurring -- in TaskWarrior.

Strongly agree on this. I'm more thinking this is a good chance to make VIT more responsive to users who have large databases. Taskwarrior (with 2.6.x) is not lightning fast for me.

In any case, I agree the cache idea is silly. I'm not convinced about moving the work for tab completion to where it's actually needed, but I'm also not volunteering to do any of the work ;).

thehunmonkgroup commented 3 months ago

In any case, I agree the cache idea is silly. I'm not convinced about moving the work for tab completion to where it's actually needed, but I'm also not volunteering to do any of the work ;).

I'm pretty sure that would be a non-trivial change, and one I'm not willing to spend time on, either.

I'm actually quite surprised that data access performance is worse in 3.x -- they are using a database, and SQLite is no slouch. It really seems there needs to be some refactoring on their data structure so data can more flexibly be pulled straight from the database.

imyxh commented 3 months ago

Something weird is going on with the task backend, because today I tried task add and it worked nearly instantly, but vit is still very very slow.

$ time task add test
Created task 103.
task add test  0.02s user 0.02s system 30% cpu 0.104 total
$ yes yes | time task del 103
Delete task 103 'test'? (yes/no) Deleting task 103 'test'.
Deleted 1 task.
task del 103  0.05s user 0.02s system 67% cpu 0.101 total

However, waiting for the Add: prompt or deleting something in vit is extremely slow, with Exec. time: 2355 ms for example. I looked through strace, and for the simple example of a, test\n, \n, qy, vit calls task a total of 14 times between the time I hit a and the time I hit q.

Specifically, it looks like this:

This seems excessive, at least for my use case. It would be nice to have a way to turn off autocomplete and all the other bells and whistles.

Btw, I had a look at the taskchampion.sqlite3 file, and it is very amusing.

sqlite> PRAGMA table_info(tasks);
0|uuid|STRING|0||1
1|data|STRING|0||0
sqlite> select * from tasks limit 1;
03fcc33c-52ae-44a3-aaf4-3af177e90bf0|{"status":"deleted","end":"1711502131","description":"test","modified":"1711502132","entry":"1711502121"}

So, task is not actually storing the data in different sql rows, it is storing it in a table with two columns, where one column is the uuid of the task, and the other is JSON for all of the task's data. So every time we e.g. query task _tags, it's not doing something like select * from tags, it has to parse the whole entire tasks table and pick out what tags exist. :skull:

thehunmonkgroup commented 3 months ago

This seems excessive

It's not. Those calls are needed to properly build both the autocomplete and the full interface display for a report. Again, this is NOT a VIT issue. task should run fast, if it's not, then your database is too big, and/or TaskWarrior 3.x has serious performance issues.

It might be possible to execute the autocomplete task commands in a separate thread, then inject the values into the autocomplete object -- this would be acceptable, as it would only mean autocomplete would not be available until those commands finished executing, which it would not be anyways in the current workflow. So I would consider a well-written PR for this change, but I probably will not do the work to implement it myself.