Naereen / uLogMe

:bar_chart: :chart_with_upwards_trend: Automatically collect and visualize usage statistics on Ubuntu, with :musical_keyboard: key frequencies and :computer: window titles →
https://Naereen.GitHub.io/uLogMe/
MIT License
67 stars 14 forks source link
d3js logging pretty quantified-self spyware visualization

uLogMe

How productive were you today? How much code have you written? Where did your time go?

Keep track of your computer activity throughout the day: visualize your active window titles and the number and frequency of keystrokes, in beautiful and responsive HTML timelines.

Current features:

The project currently only works on Ubuntu or Debian-like Linux (for an OSX version, see the original project). It uses the new fancy Promises feature of ECMAScript 6. This might not be implemented in all browsers, but recent one should have it (recent Chrome and Firefox are fine, at least).

Other self-quantified projects? I've been using cloud-based WakaTime.com since 2015 (see blog post, and locally-hosted ActivityWatch since 2021, but still use uLogMe everyday.

Screenshots

"Daily view" page

Demo - Daily view

"Overview" page

Demo - Overview Demo - Overview

Demo (by @karpathy)

See a blog post (along with multiple screenshots) describing the project here.


Getting started

To install uLogMe

  1. Clone the repository to some folder: $ git clone https://github.com/Naereen/uLogMe.git
  2. If you're on Ubuntu, make sure you have the dependencies: the xdotool xinput wmctrl xprintidle packages are required (to install them: $ sudo apt-get install xdotool xinput wmctrl xprintidle). On other Linux distribution, install them also, and you may also need gnome-screensaver ($ sudo PACKAGEMANAGER install gnome-screensaver where PACKAGEMANAGER=pacman on ArchLinux, PACKAGEMANAGER=yum on Fedora, etc).
# maybe do that, or wherever you want
cd ~/.local/

# ONLY the first time do that to install the project
git clone https://github.com/Naereen/uLogMe.git

# and ONLY ONCE run this to install the dependencies
sudo apt install xdotool xinput wmctrl xprintidle
# or use 'pacman' on ArchLinux or 'yum' on Fedora or 'brew' or a similar tool on Mac OS X

To start recording

  1. cd uLogMe/scripts inside the directory and run $ ./ulogme_data.sh. This will launch two scripts.
    • The first one, keyfreq.sh, records the frequency of keystrokes,
    • and the other one,logactivewin, records active window titles.
    • Both write their logs into log files in the logs/ directory. Every log file is very simple: just the Unix time stamp followed by data, one per line (plain text file).
    • We tried to be smart and only logging the useful data.
cd ~/.local/uLogMe/  # or wherever you installed uLogme
cd scripts/
./ulogme_data.sh   # starts collecting data !

The user interface

  1. Important. As a one-time setup, copy over the example settings file to your own copy: $ cp render/js/render_settings_example.js render/js/render_settings.js to create your own render_settings.js settings file. In this file modify everything to your own preferences. Follow the provided example to specify title mappings: A raw window title comes in, and we match it against regular expressions to determine what type of activity it is. For example, the code would convert "Google Chrome - some cool website" into just "Chrome", or "GitHub - Mozilla Firefox" into just "GitHub". Follow the provided example and read the comments for all settings in the file.
  2. Once that is set up, start the web server viewer: $ python ulogme_serve.py, and go to the provided address (by default, it is https://localhost:8443) in your browser. Hit the refresh button on top right every time you would like to refresh the results based on most recently recorded activity (it erases cache). You can also use a convenience file ulogme_serve.sh to do both: start the server, and open the web-page.
  3. If your data is not loading, try to explicitly run $ python export_events.py and then hit refresh. This could only be an issue the very first time you run uLogMe.
cd uLogMe/  # or wherever you installed uLogMe
# only once, create your own setting file
cp render/js/render_settings_example.js render/js/render_settings.js
# then launch the visualization server
cd scripts/
python ulogme_serve.py
# Open the page with Firefox, or use 'open' or 'xdg-open' or 'chromium-browser' or any recent browser
firefox https://localhost:8443/
# if needed
python exports_events.py

Bonus with tmux

Bonus: If you are using tmux

  1. The script ulogme_tmux.sh can be used to create a new tab in your current tmux session, name it "uLogMe", split it in half vertically, and then it starts the user interface script in the left column, and the data recording in the right column. Very convenient!

Hum... What is tmux?

  1. The best terminal multiplexer. Just go discover more by yourself (by @tmux).

User Interface

The user interface can switch between a single day view and an overview view by link on top. You have to hit the refresh button every time you'd like to pull in new data (and not your browser's refresh, Ctrl-R or F5, as it reads the cache by default).

Single day page

Overview page

Keyboard shortcuts


Interactive demo

Demo - Interactive GIF


Security concerns ?

Your private data should be safe with uLogMe:

Of course, this is a computer program, written by enthusiast programmers, not security experts: there is bugs, and there might be security problems. But none that we know of at least! Please file an issue if you notice a potential security threats !

How can I clean my data ?


:bug: Known issues


Explanations on the architecture

Demo - desktop notifications when refreshing

Ubuntu (or any Debian-like Linux)

uLogMe has three main parts:

  1. Recording scripts keyfreq.sh and logactivewin.sh. You probably will not touch these.
  2. Webserver: ulogme_serve.py which wraps Python's SimpleHTTPServer and does some basic communication with the UI. For example, the UI can ask the server to write a note to a log file, or for a refresh. ulogme_serve.sh helps to launch the Python web server more easily.
  3. The UI. Majority of the codebase is here, reading the .json files in /render and creating the visualizations. There are several common .js files, and crucially the index.html and overview.html files, that are simple HTML template (with a lot of Javascript in the beginning). Feel free to adapt them to your preferences. I expect that most people might be able to contribute here to add features/cleanup/bugfix.
  4. Bonus: the ulogme_tmux.sh script, if you are using tmux.

An example of the output displayed by the two parts of the "server" side (data recording and HTTP server), in two horizontal panes in tmux: Demo - colored logs

Yes, the logs are colored, from both shell and python scripts, using color.sh for bash and ansicolortags for Python.

OSX code

Not in my fork, refer to the original project


Related projects ?

For more projects, this question on Personal Productivity Stack Exchange might be also worth a look.

:scroll: License ? GitHub license

MIT Licensed (file LICENSE).

© 2014-2016 Andrej Karpathy and GitHub collaborators, and © 2016-2018 Lilian Besson and GitHub collaborators.

Maintenance Ask Me Anything ! Analytics

ForTheBadge uses-badges ForTheBadge uses-git

Contains-Pokémon : contains-Bulbasaur contains-Charmander contains-Dratini contains-pikachu contains-Snorlax contains-Squirtle

ForTheBadge uses-css ForTheBadge uses-html ForTheBadge uses-js

ForTheBadge built-with-love