Custom watcher visualization not working as expected

[Alwinator]

Hello, first of I want to congratulate you on your awesome project, which I have already installed on all my devices. I am currently working a a custom watcher for ActivityWatch that detects if I am standing or sitting (I have a height adjustable table). I will also publish it as soon as I am finished.

The problem is that the results are not shown as expected. I have taken some screenshots and wrote the things that are wrong in my opinion. By the way I am not sure if these are programming mistakes from my side or bugs in ActivityWatch. So I am sorry if I have understood something wrong.

The problem is that there is no text directly in the time lines. Only if I hover them. I would also like that sitting and standing (two different values) are in different colors.

And in Activity > Window my watcher is just missing.

I am using version 0.9.2 (latest) on Ubuntu.

Thank you in advance!

[johan-bjareholt]

Really cool that you have written such a watcher, I have had a standing desk for 6 years now and it would be cool to have such a watcher, but the past few months I have been standing 100% of the time so I guess it's become less useful for me now :smile:

The problem is that there is no text directly in the time lines. Only if I hover them. I would also like that sitting and standing (two different values) are in different colors.

This is hardcoded for different buckettypes because that's the only way for the web-ui to know which one it's supposed to display (for example, how would the web-ui otherwise know whether to change color and text on the "status" field or the "height" field in your case?). We have though about making this more configurable in the future, but for now you unfortunately need to have local patches in the web-ui to solve this. It's very easy to fix however, see this file https://github.com/ActivityWatch/aw-webui/blob/master/src/util/color.js

And in Activity > Window my watcher is just missing.

The window tab in the activity view is only supposed to show windows and table height/status is not a window so I'd consider that to work as expected :slightly_smiling_face:

[Alwinator]

Really cool that you have written such a watcher, I have had a standing desk for 6 years now and it would be cool to have such a watcher, but the past few months I have been standing 100% of the time so I guess it's become less useful for me now smile

Thank you!

This is hardcoded for different buckettypes because that's the only way for the web-ui to know which one it's supposed to display (for example, how would the web-ui otherwise know whether to change color and text on the "status" field or the "height" field in your case?). We have though about making this more configurable in the future, but for now you unfortunately need to have local patches in the web-ui to solve this. It's very easy to fix however, see this file https://github.com/ActivityWatch/aw-webui/blob/master/src/util/color.js

In my opinion it would make sense to create "default names" e.g. if there is a field in the data object which is called "display-name" it will automatically be used for the timelines and a field "color" which is just the color code. With these changes my data object would look like:

{
    "status": "sitting",
    "table-height": 80,
    "display-name": "sitting (80 cm)",
    "color": "#00FF00"
}

The window tab in the activity view is only supposed to show windows and table height/status is not a window so I'd consider that to work as expected slightly_smiling_face

Yes, that makes sense xD, but it would be great if you could create custom activity tabs. For example I insert a sitting.html (or vue component) in my watcher and then it will be loaded as activity tab.

[johan-bjareholt]

In my opinion it would make sense to create "default names" e.g. if there is a field in the data object which is called "display-name" it will automatically be used for the timelines and a field "color" which is just the color code. With these changes my data object would look like:

We will not do this for at least two reasons

• that info has nothing to do with the data itself but with the visualization, such things should not be stored so they can be easily changed later if one so chooses (such as changing the color)
• this will take up a lot of unnecessary diskspace in the database as it will be needed for every event, if we added that to buckets with 200 000 events (which is not too uncommon) it would take up a significant amount of space.

Yes, that makes sense xD, but it would be great if you could create custom activity tabs. For example I insert a sitting.html (or vue component) in my watcher and then it will be loaded as activity tab.

Injecting a HTML page would be an option, but the current idea we have for custom views is for the user to provide a custom query and then be able to visualize it with visualizations already available in the web-ui (similar to the query explorer, but hopefully with more visualizations added).

[Alwinator]

We will not do this for at least two reasons

• that info has nothing to do with the data itself but with the visualization, such things should not be stored so they can be easily changed later if one so chooses (such as changing the color)
• this will take up a lot of unnecessary diskspace in the database as it will be needed for every event, if we added that to buckets with 200 000 events (which is not too uncommon) it would take up a significant amount of space.

Yeah, you are right. Maybe it would make more sense to pass these parameters to the create bucket function as following:

self.client.create_bucket(self.bucket_id, event_type='table_state', queued=True, visualization={
    "display-name": "{status} ({table_height} cm)"
})

Or even more complex, but more flexible:

def get_display_name(data):
    return f"{data['status']} ({data['table_height']} cm)"

def get_color(data):
    if data['status'] == "sitting":
        return "#00FF00"
    return "#FF0000'"

self.client.create_bucket(self.bucket_id, event_type='table_state', queued=True, visualization={
    "display-name": get_display_name,
    "color": get_color
})

Of course the second one would be extremely complex to implement, but it would be great. Maybe it would also be easier to pass this code directly in JavaScript.

Injecting a HTML page would be an option, but the current idea we have for custom views is for the user to provide a custom query and then be able to visualize it with visualizations already available in the web-ui (similar to the query explorer, but hopefully with more visualizations added).

Okay, this also sounds good.

[johan-bjareholt]

The issue with saving it generically for all visualizations is that this is really something that might depend on context. For example if it's a window bucket we in some cases want to have the color or text depending on either the appname, the window title or even the category it has matched. So that solution will not work for that use-case.

I think the best place to solve it is in the web-ui, the server really shouldn't have to keep track of things like this.

[Alwinator]

Yeah, I think you are right, but then I would very appreciate if I could extend the web-ui with my watcher.

[johan-bjareholt]

Absolutely, I will take a look at making it easier to extend the web-ui next week. You are not the first to request this. The master branch has some changes which makes the code more modular and more easy to configure which could be reused for this case I believe.

What is most important for your use-case do you think, a summary for day/week/month or a timeline for a single day? I think the summary would be easier to fix right now due to how the UI is laid out.

[Alwinator]

Absolutely, I will take a look at making it easier to extend the web-ui next week. You are not the first to request this. The master branch has some changes which makes the code more modular and more easy to configure which could be reused for this case I believe.

Perfect, I am looking forward for the next commits. :)

What is most important for your use-case do you think, a summary for day/week/month or a timeline for a single day? I think the summary would be easier to fix right now due to how the UI is laid out.

The summary would be great. I would also appreciate it if it includes averages (I have also created a feature request about averages some days ago)

Avg. sitting 3.46h / day

A cool end result could for example also be a chart with: 37% sitting and 63% standing.

Thank you in advance!

[Alwinator]

Absolutely, I will take a look at making it easier to extend the web-ui next week. You are not the first to request this. The master branch has some changes which makes the code more modular and more easy to configure which could be reused for this case I believe.

Are there any news yet? Is it already possible to extend the web-ui?

[johan-bjareholt]

@Alwinator Unfortunately no progress. I've prioritized the migration to aw-server-rust first the past couple months.

[Alwinator]

@johan-bjareholt Okay, can you just write a message if there are news so that I get a notification? How long do you think will it approximately take?

[johan-bjareholt]

@Alwinator Sorry but I only work on ActivityWatch when I have some spare time over so we really don't have a planned schedule for different features. But once I get to work on it I'd expect it to take a couple of days to implement. But I'll keep you updated!

[Alwinator]

@Alwinator Sorry but I only work on ActivityWatch when I have some spare time over so we really don't have a planned schedule for different features. But once I get to work on it I'd expect it to take a couple of days to implement. But I'll keep you updated!

Perfect, thank you!

[ErikBjare]

@Alwinator It's not exactly what you're looking for, but it is a step in the right direction: https://github.com/ActivityWatch/aw-webui/pull/233

[Alwinator]

@ErikBjare Looks interesting, could you inform me when there is a Windows build available?

[Alwinator]

@Alwinator It's not exactly what you're looking for, but it is a step in the right direction: ActivityWatch/aw-webui#233

It is cool for users, but it is also possible to add a custom visualization (with a watcher or source code), not just the predefined ones?

[ErikBjare]

@Alwinator You can add your own visualizations by modifying https://github.com/ActivityWatch/aw-webui/blob/master/src/components/SelectableVisualization.vue, building the web UI, and copying the built files into the aw-server/static directory in your ActivityWatch installation.

[Alwinator]

@Alwinator You can add your own visualizations by modifying https://github.com/ActivityWatch/aw-webui/blob/master/src/components/SelectableVisualization.vue, building the web UI, and copying the built files into the aw-server/static directory in your ActivityWatch installation.

@ErikBjare Thank you. Will there be a way to include views in the watcher? So that I could ship a table-view with my aw-table-watcher.

[Alwinator]

@ErikBjare ?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Alwinator]

Still very important

[ErikBjare]

@Alwinator Sorry, but this hasn't really been a priority, more pressing issues to deal with.

Will there be a way to include views in the watcher?

Likely no (difficult to do). But you could write a visualization that's somewhat generic/customizable, and we could merge it to master.

[Alwinator]

Thanks for the reply! Maybe I'll do it someday, but right now I'm pretty busy, unfortunately.

[Alwinator]

Thanks for the reply! Maybe I'll do it someday, but right now I'm pretty busy, unfortunately.

I found time to implement it. I will work on it in the next few days.

[Alwinator]

I am done! It was a tough amount of work, but it works. I tried to implement it as best as I could, without my Vue knowledge.

How does it work?

I created a Flask blueprint that uses the Manager from aw-qt to find watchers. All watchers that contain a static directory will be available for the custom visualization feature. Then, my blueprint includes a get_data POST route which will be called by the web-UI with the time period as a parameter. My endpoint returns the data from all (custom visualization) supported watchers as JSON. Then, this data as well as the watcher and view name will be transferred to my CustomWatcher Vue component. This component is based on an IFrame and shows the static content of the watcher directories using a different endpoint. My endpoint hands the data and the view name over to the static content of the watcher as GET parameter. With this approach, the watcher developer can use Vue, React, or Vanilla JavaScript for their watcher visualizations.

The first watcher with virtualization support is my aw-watcher-table. It uses just Vanilla JavaScript to make it as simple as possible.

It would be beautiful to separate the watcher visualization completely so that also the AFK and window watcher visualization parts are in the watchers themselves.

I hope you like my contribution.

[Alwinator]

@ErikBjare Before merging, please take a detailed look at my code. I tested it as well as I could, but I am still not 100% sure if I integrated everything in your code the right way. Please also make sure to merge all four Pull Requests at once because they depend on each other. Thanks again for your great software. I hope this software will always be maintained and never die out.

[ErikBjare]

Wow, that's really cool! Impressed that you managed to make it extensible like that, kudos! :1st_place_medal:

Might take a while for me to get around to reviewing & merging it (still busy :disappointed:).

But from a quick glance: my only gripe is that the entire manager.py gets moved into aw-core (trying to keep aw-core as lean and well-tested as possible). I'd prefer it if you could keep it in aw-qt, and perhaps do the static directory lookup in aw-server directly (or maybe passed by config?). Ideally, the watcher executable could exist not only in the installation directory, but anywhere in the PATH (and the static directory perhaps somewhere else, like /usr/share on Unix systems).

Keep in mind that the same changes will have to be made in aw-server-rust, so keeping the static directory lookup simple would help a lot (perhaps an argument in favor of passing it by config).

[ErikBjare]

Would also welcome a review by @johan-bjareholt

[Alwinator]

Wow, that's really cool! Impressed that you managed to make it extensible like that, kudos!

Thanks! :)

Might take a while for me to get around to reviewing & merging it (still busy).

No problem. I am usually busy too, so I can understand that. 😉

But from a quick glance: my only gripe is that the entire manager.py gets moved into aw-core (trying to keep aw-core as lean and well-tested as possible). I'd prefer it if you could keep it in aw-qt, and perhaps do the static directory lookup in aw-server directly (or maybe passed by config?). Ideally, the watcher executable could exist not only in the installation directory, but anywhere in the PATH (and the static directory perhaps somewhere else, like /usr/share on Unix systems).

Keep in mind that the same changes will have to be made in aw-server-rust, so keeping the static directory lookup simple would help a lot (perhaps an argument in favor of passing it by config).

Makes sense, I will improve my code. It would be nice if you can implement the rust part after I finished the feature for the Python server because I prefer Python :)

[Alwinator]

@ErikBjare I have moved the manager.py back to aw-qt and implemented a new way. Now the aw-qt passes the dict with the custom visualization watchers as an argument in JSON format. I also fixed some small visualization bugs in the Web UI. From my side, it is ready to merge now. Please don't forget to test. I am not in hurry, so if you are busy now you can merge and test it whenever you want. :)

[johan-bjareholt]

This is not at all the architecture I had in mind when I was thinking about how this could be solved earlier. I think the two big negatives about this solution is

1. Making the custom watcher visualization a completely separate webpage inside an iframe. Sure it's flexible, but treating custom visualizations as completely different webpages will make those feel like a second-class citizen. I'd prefer if we had something more integrated solution that built on-top of our already existing visualization tools and to improve and extend those for more use-cases. Like maybe just having some JSON data containing one or more types of visualization (e.g. barchart), a query and maybe if the query contains multiple types of results also what key in that result to use for each visualization. That would make custom visualizations possible to both work and look the same as the built-in visualizations. I think having a common base for both built-in watchers and third-party ones could help both by upstreaming new or improved visualizations.
2. Adding more logic to aw-qt. I like the fact that aw-qt is dead stupid and just keeps track on what's running. To me it would make more sense for the watchers to tell aw-server when they are registering themselves (or possibly shortly after) that it has some kind of visualization and to add it. By adding more logic to aw-qt we are essentially forcing users to use aw-qt for some features, which would extend what aw-qt is responsible today and might become confusing on platforms where aw-qt cannot run, such as on GNOME and other DEs which don't support tray icons.

[Alwinator]

@johan-bjareholt I understand your arguments, but

1. There will always be specific watchers which want to use a different chart than available. I would recommend your visualization concept in addition to mine that users with simple requirements can use your visualization concept and the advanced users can use my iFrame concept.

2. If there are more watchers, Activity Watch is better, so I think it is a good idea to give people the freedom to choose their own frontend framework instead of forcing them to use Vue. Forcing Vue might prevent e.g. React Developers from writing watchers. Additionally, you have less work to do because you don't need to review all the Pull Requests of the new visualizations.

3. If you want the same design for custom watchers, you could publish Activity Watch Design Guidelines or a CSS library.

4. I am sure there are ways to improve my visualization, but please merge it for now. It is better than nothing.

[ErikBjare]

I agree with @johan-bjareholt's points, in particular:

I'd prefer if we had something more integrated solution that built on-top of our already existing visualization tools and to improve and extend those for more use-cases. Like maybe just having some JSON data containing one or more types of visualization (e.g. barchart), a query and maybe if the query contains multiple types of results also what key in that result to use for each visualization.

We might get this done, eventually (somewhat like what's in Zenobase), but the reality is that it's a fair bit of work that hasn't really started, and this approach does seem more flexible and gives watcher-developers the ability to ship visualizations separately (which I think is neat, and might be very useful!).

The question now is if it makes sense to merge an approach like this in the meantime, to allow for watchers to ship some of their own (simple) visualizations without having to get them into aw-webui (and the Vue-tyranny + slow review process that follows), to @Alwinator's point.

However, the only current custom visualization that exists (in aw-watcher-table) could very easily be added as a part of aw-webui (or by making just a couple things on the summary/horizontal barchart visualization configurable).

I'm still a bit unsure how this custom_static feature would actually get used in practice if it was merged (although I can imagine a lot of ways it could be used, I'm not sure if they make sense, and if there are limitations I haven't considered).

Forcing Vue might prevent e.g. React Developers from writing watchers.

In my experience, it doesn't really (although it may scare some off). Most visualizations are just D3.js + some data wrangling in a thin Vue wrapper. There isn't really much Vue involved.

If you want the same design for custom watchers, you could publish Activity Watch Design Guidelines or a CSS library.

That's a lot of work we'd rather avoid. I think @johan-bjareholt's point is that with configurable (although not as "custom") visualizations in aw-webui, you get it basically for free.

I am sure there are ways to improve my visualization, but please merge it for now. It is better than nothing.

We don't want this to become tech debt that we might have to yank/rewrite later (much to the disappointment of those who might've come to rely on it). Since we also have to implement the server part in both aw-server and aw-server-rust we should be careful to not merge something for one server but not in another (to not drift on feature parity).

Let's try to get this right from the start and not rush a merge.

Thanks for your work on this so far @Alwinator! It might take some time/polish to get this done, but this has spurred a lot of good discussion and thoughts about how to enable custom and configurable visualizations.

To summarize current status of PRs:

• https://github.com/ActivityWatch/aw-server/pull/83
  • Just reviewed, could be simplified/generalized further + some comments on config.
• https://github.com/ActivityWatch/aw-webui/pull/301
  • Haven't reviewed fully.
• https://github.com/ActivityWatch/aw-core/pull/104
  • Closed, rewritten such that no longer needed.
• https://github.com/ActivityWatch/aw-qt/pull/78
  • Closed, not needed.

[Alwinator]

@ErikBjare Based on your comment I am not really sure if you will merge it. Should I rewrite my code based on your code review or will you never merge it? By the way, I know that the visualization in the table watcher is simple, but it was just for testing purposes. When the feature is final and merged I will probably also create more complex visualizations.

[ErikBjare]

@Alwinator I think there's a good chance we'll merge it as a general custom_static feature (that may find other applications as well), so don't lose hope!

I'd really appreciate it if you continued to work on the open PRs :)

[Alwinator]

@ErikBjare Thanks, that's relieving. Then I will continue working on this feature.

[Alwinator]

@ErikBjare I have resolved all your code review comments except for the one with the get_data endpoint. I am waiting for the aw-WebUI code review for this one.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.