Large volume of "unknown"s and slow statistics loading

[renyuneyun]

Hi. First of all, thanks very much for developing this! I'm a Wayland KDE Plasma user, and had been having issues with activitywatcher's inability in Wayland ever since I transfer to Wayland from X11. Your watcher made that working again!

However, in the meantime, I also found some strange behaviours in the log. In particular, there is now a large chunk of "unknown"s (both in Applications and Titles), as in this screenshot:

Based on my activity on that day, I believe a large chunk of the unknowns is firefox (I have firefox and firefox-dev both installed). Do you have any clue why this behaviour?

By the way, after having awatcher, I notice the loading time for ActivityWatch data views is significantly longer. Had I better open another issue for this?

[2e3s]

Based on my activity on that day, I believe a large chunk of the unknowns is firefox (I have firefox and firefox-dev both installed). Do you have any clue why this behaviour?

This looks like XWayland, KDE gives empty metadata for some apps. It would be a wrong environment detection.

1. What are the watcher's output on starting if you run it with highest verbosity -vvv? It gets to detect the environment when starting up until INFO watchers::watchers] Starting active window watcher.
2. Also, with -vvv, try to switch between the "unknown" app to a known app back and forth with that setting, it should output this log. I guess it'll be an empty caption for the unknown app in your case.
3. Do you use v0.2.1 or v0.2.0? I added some heuristics at v0.2.1 to determine X11.

By the way, after having awatcher, I notice the loading time for ActivityWatch data views is significantly longer. Had I better open another issue for this?

Do you use the bundle, do you mean that it loads the statistics page in browser for longer than if having aw-server-rust running separately, or just having the watcher within aw-server-rust? Or even aw-server is faster (the original python server)?

[renyuneyun]

Thanks for the info.

This looks like XWayland,

I have been wondering this as well, since I noticed your comment in the README about behaviour of Wayland+KDE. However, firefox is running under Wayland. I verified this through xprops and xeyes -- both did not respond to my mouse in firefox.

What are the watcher's output on starting if you run it with highest verbosity -vvv?

Please see the code block:

❯ awatcher -vvv            
[2023-11-21 11:33:53.295430 INFO awatcher] Sending to server localhost:5600
[2023-11-21 11:33:53.295470 INFO awatcher] Idle timeout: 180 seconds
[2023-11-21 11:33:53.295476 INFO awatcher] Idle polling period: 5 seconds
[2023-11-21 11:33:53.295480 INFO awatcher] Window polling period: 1 seconds
[2023-11-21 11:33:53.340342 INFO watchers::watchers] Selected wl_kwin_idle::IdleWatcher as idle watcher
[2023-11-21 11:33:53.340361 INFO watchers::watchers] Starting idle watcher
[2023-11-21 11:33:53.346432 DEBUG watchers::watchers] wl_foreign_toplevel::WindowWatcher cannot run: the requested global was not found in the registry
[2023-11-21 11:33:53.352552 DEBUG watchers::watchers::kwin_window] Starting KWin script 3
[2023-11-21 11:33:53.357338 INFO watchers::watchers] Selected kwin_window::WindowWatcher as active window watcher
[2023-11-21 11:33:53.357356 INFO watchers::watchers] Starting active window watcher

So, firefox window is correctly detected.

, try to switch between the "unknown" app to a known app back and forth with that setting

There is no unknown yet (after ~20min). I'll keep an eye on my further activities today. But at least firefox is not.

Do you use v0.2.1 or v0.2.0? I added some heuristics at v0.2.1 to determine X11.

I'm using rev 0ccdba4, using the awatcher-git package on AUR (ignore it's version code since it's actually recalculated while building; I'm on archlinux). So it's v0.2.1. Though, awatcher --version still show v0.2.0.

Do you use the bundle, do you mean that it loads the statistics page in browser for longer than if having aw-server-rust running separately, or just having the watcher within aw-server-rust? Or even aw-server is faster (the original python server)?

I'm using the separated watcher. I have aw-server executed on its own (actually auto-start-up, and I didn't find out how to disable that), and then start awatcher separately from a terminal.

I'm not entirely sure which aw-server I'm using, as I directly installed the activitywatch-bin package from AUR. It's probably the Python server?

Yes, I mean the time needed for loading the statistics page in the browser is longer, compared to before having used awatcher. It used to load within 1s. Now it takes a few seconds. Maybe the speed difference is related to the volume of data in the database, related to how awatcher stores data? Just a guess.

[renyuneyun]

Actually, the unknown could be (slightly) related to AW's built-in watchers and/or combination of AFK. Before using awatcher, the statistics shows unknowns as well, but with much less time span (e.g. <1h per day). I presumed that was related to it's incapability of Wayland+KDE.

But, as you see from the earlier screenshot, there are much more records now, and also much more time labeled as unknown...

Actually, AFK detection is better than previously, but not entirely solved. When I look at the timeline just now, I found that AFK detection is incorrect. I have been very actively using the machine just now, for ~20 min, but it's recorded as AFK in the timeline. I presume awatcher also replaces AW's own AFK watcher? (The lack of control and information in AW UI is really an issue...)

[2e3s]

Selected kwin_window::WindowWatcher

So the environment is detected correctly. KWin script doesn't report empty or unknown window titles in my experience, it reports exactly what KDE/KWin knows about the app window. Maybe sometimes some application could be the culprit.

Actually, the unknown could be (slightly) related to AW's built-in watchers and/or combination of AFK.

Have you turned off the original AW watchers for active window and AFK in AW config? The original watcher understands only X11, so it would use XWayland with reported unknowns. You can see if the original watchers by running in CLI: ps aux | grep "watcher", there would be aw-watcher-window and aw-watcher-afk.

I have to write a better instruction how to use the watcher within the ActivityWatch distribution in README, that's my neglect.

Just for a reference, the watchers a turned on and off by aw-qt (the tray icon), there is a submenu for it. The original config is at ~/.config/activitywatch/aw-qt/aw-qt.toml, watchers can be enabled and disabled with it as well.

I'm using the separated watcher. I have aw-server executed on its own (actually auto-start-up, and I didn't find out how to disable that), and then start awatcher separately from a terminal.

So I don't think the slower speed can be related to using awatcher. It does the same job as the original watchers, and I think I have copied timings from their configs, so it gotta be the same volume of data. Maybe it's related to non-disabled original AW watchers, so 2x more data is reported.

[renyuneyun]

After today's activities, I got some more information. That shows some clues, but does not answer the initial questions.

Here is what I got in the recent hour(s), in the timeline:

So here are some critical observations:

• Events are somehow made incontinuous. I am certain I have been using firefox for that large chunk of time (the yellow part)
• Those "inserted" incontinuity are "unknown"s
• Those red small chunks (to the left) are also unknows. I was not using my laptop at that time, so would expect nothing is recorded in fact (or the last window I was on)
• AFKs are often wrong (see the log below and comparison with screenshot)

Here is the log from awatcher -vvv from 17:06 to 17:31, during which I had a short leave, screen locked automatically, and I returned: (Some fields in window title replaced with [REMOVED].)

[2023-11-21 17:06:52.492589 DEBUG watchers::watchers::kwin_window] Active window class: "firefox", name: "firefox", caption: "[REMOVED] - Google 表格 — Mozilla Firefox"
[2023-11-21 17:13:58.603156 DEBUG watchers::watchers::wl_kwin_idle] Idle
[2023-11-21 17:13:58.603181 DEBUG watchers::watchers::wl_kwin_idle] Reporting as changed to idle
[2023-11-21 17:15:58.908926 DEBUG watchers::watchers::kwin_window] Active window class: "kwin_wayland", name: "kwin_wayland", caption: ""
[2023-11-21 17:15:58.931798 DEBUG watchers::watchers::kwin_window] Active window class: "kwin_wayland", name: "kwin_wayland", caption: ""
[2023-11-21 17:16:03.384165 DEBUG watchers::watchers::kwin_window] Active window class: "firefox", name: "firefox", caption: "[REMOVED] - Google 表格 — Mozilla Firefox"
[2023-11-21 17:16:04.599537 DEBUG watchers::watchers::wl_kwin_idle] Resumed
[2023-11-21 17:16:04.599584 DEBUG watchers::watchers::wl_kwin_idle] Reporting as no longer idle
[2023-11-21 17:31:43.659632 DEBUG watchers::watchers::kwin_window] Active window class: "firefox", name: "firefox", caption: "[REMOVED2] - Google 表格 — Mozilla Firefox"
[2023-11-21 17:31:55.319480 DEBUG watchers::watchers::kwin_window] Active window class: "firefox", name: "firefox", caption: "[REMOVED] - Google 表格 — Mozilla Firefox"

[renyuneyun]

Ah, what a coincidence in writing comments :)

Have you turned off the original AW watchers for active window and AFK in AW config? The original watcher understands only X11, so it would use XWayland with reported unknowns. You can see if the original watchers by running in CLI: ps aux | grep "watcher", there would be aw-watcher-window and aw-watcher-afk.

This is probably the reason. I did not explicitly turn them off. I was assuming awatcher would replace them (or suppress them) since awatcher also records to the AFK and window buckets. It looks like I was wrong.

[2e3s]

Okay, thank you. I see how it can be confusing. I have added instructions: https://github.com/2e3s/awatcher#module-for-activitywatch And pre-released with fixed version and to match the instructions: https://github.com/2e3s/awatcher/releases/tag/v0.2.2-beta2

[renyuneyun]

After turning down AW's built-in AFK and window watchers, and running for a few days, everything seems to be normal now, including the loading speed. So I presume the loading speed was due to AW trying to figure out events for concurrent/conflicting logs.

By the way, thanks for the instruction. I haven't tested loading awatcher with autorun with aw-qt yet. But a quick question: what is the rule for specifying the watcher module in aw-qt.toml? Like, my package provides /usr/bin/awatcher. Should I specify awatcher (i.e. an executable within PATH which happens to be an AW watcher module) there?

[2e3s]

Like, my package provides /usr/bin/awatcher. Should I specify awatcher (i.e. an executable within PATH which happens to be an AW watcher module) there?

Thank you for supporting the package. Please, rename the binary to aw-awatcher. See my recent commit https://github.com/2e3s/awatcher/commit/c6a5df58d8ba7c6f92b5f08d03018f92de417051 for the reference. I've split more explicitly aw-awatcher in Cargo.toml in order to build aw-awatcher_*.deb. The point of having aw- in the beginning of executable file is to be discovered by aw-qt app from the original distribution. I've just implemented the module-discovering functionality in the bundle the same way as in aw-qt, hence finally sorted out namings.

[2e3s]

The name with aw- in $PATH is enough to appear in the tray list of modules AFAIU. My bundled tray can enable the autostart from the UI, aw-qt from ActivityWatch distribution seems to require to add aw-awatcher to aw-qt.toml in order to be autostarted.

[renyuneyun]

Thanks for the explanations! I have attempted to config aw-qt, and led to another issue #5 . Anyway, this issue is resolved. Conclusion: one should not be running aw-watcher-afk and aw-watcher-window (both are built-in watchers to AW) simultaneously with awatcher.