Idea: Buffering and inserting at interval to improve performance

[BOLL7708]

Is your feature request related to a problem? Please describe. Typing in the editor becomes quite sluggish when the database surpasses ~10MB, which took me less than two months.

Describe the solution you'd like

1. Buffering input and editor actions in some manner to be inserted asynchronously at an interval.
2. A rotating database that can switch to a new one every day/week/month.

Describe alternatives you've considered

1. For me to manually rotate the database out and possibly merge them at a later date.
2. To deactivate the plugin until it is updated. (this is what I currently did)

Additional context I usually run multiple instances of JetBrains editors at the same time, often five but sometimes up to ten instances. This is a mix of Android Studio, Rider and PHPStorm, all in which I used this plugin.

Thanks for making it, it has worked really well, just with this performance caveat!

[jpv-os]

Thank you for your input. I will look into it and get back to you soon. As a quick fix, you could copy your local database file (see the plugin settings for your database location) and rotate the DB manually by either renaming/moving your current DB or changing the DB location to a new file.

[jpv-os]

I decided to implement the configurable insert interval, which means that the plugin will now collect all actions within a certain time interval (default 1 second) and then persist the buffer.

While working on it, I realised that the tool window will probably have a much bigger performance impact when working with a large dataset. Before the update, the tool window would query the database at a fixed interval of 1 second to build the statistics for the charts. I also made this interval configurable, so that you can adjust it to your needs.

After installing the new update, please try using it for a bit with the default settings to see if your performance problem is already solved with the new input batching. Then, if it didn't fully solve your issue, try increasing the update interval.

I also considered your idea of using rotating databases, but I have not decided how I would like to implement the functionality and what the core idea of a rotating database should be (organisation of data by splitting it per week/day/..., or maximise performance by splitting after a certain size threshold, ...?).

The update is now submitted for approval by JetBrains and will usually go public within a day or two. Let me know wether the update works for you and if we can further improve upon it.

Below you find a screenshot of the relevant section of the settings page:

Edit: I just noticed a typo in the settings hint text, will be fixed soon.

[BOLL7708]

Looking forward to test this!

Regarding the DB rotation, that was just a second option, not nothing I actually need or want, but one way to solve this. Personally I prefer it just getting dumped into the one place as I want to run queries on the full set 😁

Regarding the tool window, I only go into it manually so when it's not visible maybe it can skip updating entirely? And then update when shown? Not sure if that is possible but it's an idea.

And this was super quick, cheers! 😃

[jpv-os]

I am still learning how to use the Intellij platform UI components, so I want to investigate how I can skip the updates to the tool window while it is not visible, but I am not sure how it can be achieved at the moment. I will add it to the plugin development roadmap in any case.

[jpv-os]

Did the update fix your issue @BOLL7708 ?

[BOLL7708]

I haven't yet applied it at work, but I run the latest version at home now, but at home I hadn't bumped into this issue yet as my database is not as big. Hobby coding is naturally less than work coding 😅 I'll re-enable the plugin and update it when I'm back in the office tomorrow! 😁

[BOLL7708]

Some bad news, I updated and activated the plugin at work, I set the write interval to five minutes and the tool panel to update every hour.

I had three instances of PHPStorm running that I restarted after updating the plugin, and then I started working again.

What happened next was that I was using one editor, then switched to another and opened the tool panel for this plugin. This hung all three clients. None of them reacted to clicks or scrolling, and eventually I terminated the bunch.

I'm not sure what is going on, but clearly something is amiss when using a big delay and using multiple editors at the same time. 😬 As I had to get work done, I deactivated the plugin again, but I can do more testing if you need me to.

[jpv-os]

Thanks for the feedback, I will do some testing and report back in a short while. I have already tested the plugin with multiple instances open, but it appears there is something I may have missed. I will do some more in depth testing and logging to find out what's going on.

[jpv-os]

No luck yet with finding the cause. Could you provide the settings that you used, please? Especially your idle timeout, update interval and insert interval. Thank you very much!

[BOLL7708]

Action database
  Path to the SQLite database file: .keylogger-plugin\db.sqlite
  ✅ Use relative path from home directory
  Idle timeout: 1000
  Insert interval: 300000
Tool window
  Update interval: 3600000
  Action history size: 25
IdeaVim
  ❌ Enable compatibility mode

I enabled the plugin again today and did not get the same hang, yet, I guess I was really unlucky the other day 😅 I'll run with it next week and see how it fairs!

[jpv-os]

Thanks for your feedback. So far I haven't had time to look into it even further, so please report back if you encounter any problems again.

[BOLL7708]

As a side note, I think I bungled my own testing some, as I updated the settings and forgot to click apply, so the next time I got in there they were reset 😅 I indeed haven't bumped into the issue I only had the very first time again, so it feels solid! 😁 Cheers!