search

google / taqo-paco

Desktop and new mobile client apps for the Paco user behavior research platform (https://www.pacoapp.com/).
Apache License 2.0
12 stars 7 forks source link

App Logger Linux - Using Wayland misses some app info #41

Open taqo-issues-bot opened 3 years ago

taqo-issues-bot commented 3 years ago

Issue by BobEvans Wed Jun 17 05:40:22 2020

Not sure what the cause for some apps not getting logged.

I can see window focus events for Taqo and Firefox as I go back and forth between them but not for Terminal, or LibreOffice.

Maybe this has something to do with xprops sampling?

Also, it seems to be missing some tab focus events in Firefox.

taqo-issues-bot commented 3 years ago

Comment by BobEvans Wed Jun 17 08:04:49 2020

As a little test, I tried using xdotool in a loop to inspect the current window

xdotool getactivewindow getwindowname

It returns empty for Terminal and libreoffice, but, it does catch each firefox tab focus.

So, it seems like some apps do not report property through the X windows apis? Maybe?

taqo-issues-bot commented 3 years ago

Comment by BobEvans Wed Jun 17 08:30:54 2020

It seems like it might be related to x windows and wayland windows.

taqo-issues-bot commented 3 years ago

Comment by BobEvans Wed Jun 17 08:47:10 2020

Found some threads on it including this example using gdbus:

https://gist.github.com/rbreaves/257c3edfa301786e66e964d7ac036269

found in this thread (https://github.com/ActivityWatch/activitywatch/issues/92)

taqo-issues-bot commented 3 years ago

Comment by mikemaksymowych Wed Jun 17 15:12:00 2020

Wayland would be an issue, thanks. Which terminal are you using? For me with Xfce4-Terminal, it does work

taqo-issues-bot commented 3 years ago

Comment by mikemaksymowych Wed Jun 17 15:33:29 2020

On my Debian 10 I tried a few different terminals and openoffice and xdotool seemed to work with Xfce4, Cinnamon, and Gnome.

But if I start Gnome to use Wayland explicitly, then pretty much everything fails.

taqo-issues-bot commented 3 years ago

Comment by BobEvans Wed Jun 17 19:13:04 2020

using the gdbus inspection commandline, it returns "gnome-terminal-server" for the terminal. It is the terminal app that is in the apps list in the gdm3 window manager.

taqo-issues-bot commented 3 years ago

Comment by mikemaksymowych Fri Jun 19 15:32:12 2020

For the firefox tabs, I think the issue may be that I'm only tracking the app name (not the window title) as a comparison for if a new app has gained focus. Since all firefox tabs would have the same app name ("firefox" or something like that), they're not being considered. That will be an easy fix.

For Wayland, it will be trickier. It's not Wayland that we need to hook but rather the Wayland compositor, of which there are many. Weston is the reference compositor. Mutter is from Gnome, as in the example you found. In general: https://wiki.archlinux.org/index.php/Wayland#Compositors

My guess would be to support Gnome, KDE, and sway, in that order, although I'm not sure how to identify which one is running yet (maybe XDG env vars)

taqo-issues-bot commented 3 years ago

Comment by BobEvans Fri Jun 19 18:03:10 2020

I thought that might be the case with Firefox. Cool.

On Wayland, so much fun! :-)

I wonder if that gdbus monitor snippet will work in all these environments? It may not give us everything but at least the app name.

My intuition is that we target our primary workstation participants, and add new ones as needed. It's not uncommon, I think, for people to pick their own window system as part of customizing their setup,e.g. some people love tiling window managers.

taqo-issues-bot commented 3 years ago

Comment by mikemaksymowych Fri Jun 19 18:22:36 2020

The DBus snippet explicitly calls Gnome Shell, so I'd guess it will only work with Gnome.

It's not so much the window managers as the back-ends, e.g. X11 or some Wayland compositor, and there should be a limited number of those. For example, i3 uses X11, I think that's the most common tiling WM. So we should have that covered.

Wayland is worse than X11 as there seem to be a lot of compositors. sway is the Wayland equivalent of i3 and it also implements its own compositor.

taqo-issues-bot commented 3 years ago

Comment by BobEvans Fri Jun 19 19:40:17 2020

Oh, right. :-)

Lots of questions about the current state of windowing on linux.

taqo-issues-bot commented 3 years ago

Comment by BobEvans Tue Jun 23 07:58:28 2020

So, I filed a bug with Flutter (https://github.com/flutter/flutter/issues/60032) based on the latest gtk embedder which throws an error to the effect of "Not a Native X Window" on Taqo launch.

It seems like the problem is Wayland based on this Flutter issue: https://github.com/flutter/flutter/issues/57932

Since it seems to work on all X based environments, and since our first target participants all use X based environments, I think we should de-prioritize this until Flutter supports Wayland or we need to study people using Wayland. (How many people is that?)

taqo-issues-bot commented 3 years ago

Comment by BobEvans Tue Jun 23 08:39:15 2020

One other finding. If running in a Wayland environment, telling GTK to use X (and then, I am guessing due to XWayland) will make it work. Example: env GDK_BACKEND=x11 taqo

taqo-issues-bot commented 3 years ago

Comment by mikemaksymowych Tue Jun 23 14:00:34 2020

Sounds good.