many keyfreq.sh processes

alperyilmaz commented 3 years ago

first of all, thanks for this great tool. I was using the karpathy/ulogme which required sudo for keyfreq, your version does not need it and it's much appreciated. When I run ps -ef f I see many copies of keyfreq.sh in the output. Is this expected or something is off in my case?

Initially, I thought the problem might be too many tty/terminals being open. But when I closed many of them the number of keyfreq.sh entries in ps output didn't change.

Luckily, there are no duplicate entries in logs/keyfreq_today.txt file.

Here's process list:

/bin/bash /home/user/softwares/uLogMe/scripts/ulogme_data.sh
  \_ /bin/bash ./keyfreq.sh
  |   \_ /bin/bash ./keyfreq.sh
  |   |   \_ /bin/bash ./keyfreq.sh
  |   |       \_ timeout -s 10 10 xinput test 5
  |   |       |   \_ xinput test 5
  |   |       \_ grep press
  |   |       \_ tr -dc 0-9\n
  |   |       \_ grep -vE 37|50|62|64|66|105|204
  |   |       \_ tr -c \n 0
  |   \_ /bin/bash ./keyfreq.sh
  |   |   \_ /bin/bash ./keyfreq.sh
  |   |       \_ timeout -s 10 10 xinput test 11
  |   |       |   \_ xinput test 11
  |   |       \_ grep press
  |   |       \_ tr -dc 0-9\n
  |   |       \_ grep -vE 37|50|62|64|66|105|204
  |   |       \_ tr -c \n 0
  |   \_ /bin/bash ./keyfreq.sh
  |   |   \_ /bin/bash ./keyfreq.sh
  |   |       \_ timeout -s 10 10 xinput test 16
  |   |       |   \_ xinput test 16
  |   |       \_ grep press
  |   |       \_ tr -dc 0-9\n
  |   |       \_ grep -vE 37|50|62|64|66|105|204
  |   |       \_ tr -c \n 0
  |   \_ /bin/bash ./keyfreq.sh
  |   |   \_ /bin/bash ./keyfreq.sh
  |   |       \_ timeout -s 10 10 xinput test 17
  |   |       |   \_ xinput test 17
  |   |       \_ grep press
  |   |       \_ tr -dc 0-9\n
  |   |       \_ grep -vE 37|50|62|64|66|105|204
  |   |       \_ tr -c \n 0
  |   \_ /bin/bash ./keyfreq.sh
  |   |   \_ /bin/bash ./keyfreq.sh
  |   |       \_ timeout -s 10 10 xinput test 8
  |   |       |   \_ xinput test 8
  |   |       \_ grep press
  |   |       \_ tr -dc 0-9\n
  |   |       \_ grep -vE 37|50|62|64|66|105|204
  |   |       \_ tr -c \n 0
  |   \_ /bin/bash ./keyfreq.sh
  |   |   \_ /bin/bash ./keyfreq.sh
  |   |       \_ timeout -s 10 10 xinput test 13
  |   |       |   \_ xinput test 13
  |   |       \_ grep press
  |   |       \_ tr -dc 0-9\n
  |   |       \_ grep -vE 37|50|62|64|66|105|204
  |   |       \_ tr -c \n 0
  |   \_ /bin/bash ./keyfreq.sh
  |   |   \_ /bin/bash ./keyfreq.sh
  |   |       \_ timeout -s 10 10 xinput test 7
  |   |       |   \_ xinput test 7
  |   |       \_ grep press
  |   |       \_ tr -dc 0-9\n
  |   |       \_ grep -vE 37|50|62|64|66|105|204
  |   |       \_ tr -c \n 0
  |   \_ /bin/bash ./keyfreq.sh
  |   |   \_ /bin/bash ./keyfreq.sh
  |   |       \_ timeout -s 10 10 xinput test 18
  |   |       |   \_ xinput test 18
  |   |       \_ grep press
  |   |       \_ tr -dc 0-9\n
  |   |       \_ grep -vE 37|50|62|64|66|105|204
  |   |       \_ tr -c \n 0
  |   \_ /bin/bash ./keyfreq.sh
  |   |   \_ /bin/bash ./keyfreq.sh
  |   |       \_ timeout -s 10 10 xinput test 10
  |   |       |   \_ xinput test 10
  |   |       \_ grep press
  |   |       \_ tr -dc 0-9\n
  |   |       \_ grep -vE 37|50|62|64|66|105|204
  |   |       \_ tr -c \n 0
  |   \_ /bin/bash ./keyfreq.sh
  |       \_ /bin/bash ./keyfreq.sh
  |           \_ timeout -s 10 10 xinput test 6
  |           |   \_ xinput test 6
  |           \_ grep press
  |           \_ tr -dc 0-9\n
  |           \_ grep -vE 37|50|62|64|66|105|204
  |           \_ tr -c \n 0
  \_ /bin/bash ./logactivewin.sh
      \_ sleep 2

I checked the shared memory folder and saw the following list:

$ ls /dev/shm/keyfreqraw.*
 /dev/shm/keyfreqraw.10   /dev/shm/keyfreqraw.16   /dev/shm/keyfreqraw.19   /dev/shm/keyfreqraw.7
 /dev/shm/keyfreqraw.11   /dev/shm/keyfreqraw.17   /dev/shm/keyfreqraw.5    /dev/shm/keyfreqraw.8
 /dev/shm/keyfreqraw.13   /dev/shm/keyfreqraw.18   /dev/shm/keyfreqraw.6

And here's the list of inputs:

$ xinput
⎡ Virtual core pointer                      id=2    [master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer                id=4    [slave  pointer  (2)]
⎜   ↳ MSFT0001:01 06CB:CD5F Mouse               id=14   [slave  pointer  (2)]
⎜   ↳ MSFT0001:01 06CB:CD5F Touchpad            id=15   [slave  pointer  (2)]
⎜   ↳ ITE Tech. Inc. ITE Device(8910) Consumer Control  id=12   [slave  pointer  (2)]
⎜   ↳ Logitech M315/235/317                     id=9    [slave  pointer  (2)]
⎣ Virtual core keyboard                     id=3    [master keyboard (2)]
    ↳ Virtual core XTEST keyboard               id=5    [slave  keyboard (3)]
    ↳ ITE Tech. Inc. ITE Device(8910) Keyboard  id=11   [slave  keyboard (3)]
    ↳ Ideapad extra buttons                     id=16   [slave  keyboard (3)]
    ↳ AT Translated Set 2 keyboard              id=17   [slave  keyboard (3)]
    ↳ Power Button                              id=8    [slave  keyboard (3)]
    ↳ ITE Tech. Inc. ITE Device(8910) Wireless Radio Control    id=13   [slave  keyboard (3)]
    ↳ Video Bus                                 id=7    [slave  keyboard (3)]
    ↳ ITE Tech. Inc. ITE Device(8910) Consumer Control  id=18   [slave  keyboard (3)]
    ↳ Integrated Camera: Integrated C           id=10   [slave  keyboard (3)]
    ↳ Video Bus                                 id=6    [slave  keyboard (3)]

Naereen commented 3 years ago

Hi @alperyilmaz! Thanks for the appreciations, and for your detailed feedback! It's perfectly normal to see several processes of these xinput test [kbd_id], see in the keyfreq.sh script there is a for loop:

    # check each possible keyboard
    keyboardIds=$(xinput | grep 'slave  keyboard' | grep -o 'id=[0-9]*' | cut -d= -f2)
    for kbd_id in $keyboardIds; do
     ...

As you saw, there is a few keyboards on your device, and keyfreq.sh is very naive and constantly try them all. If you look at your pstree output above, you will see that there is only on mother process of keyfreq.sh, showing that it was only launched once. As soon as there is timeout in the for loop (the one I show just above), pstree will show different processes being active simultaneously.

So, to conclude, yes everything works as expected! I'm sure the keyfreq.sh could be improved, for instance by storing somewhere the list of kbd_id of active keyboards and only scanned these (which for most laptops will be only real keyboard!), but I never tried to optimize this. One reason is that on my laptops, there is always a background uLogMe process running, and if I change from one office to another one, I use different external keyboards on each office, so I don't want to have to manually relaunch uLogMe!

alperyilmaz commented 3 years ago

Thanks for detailed answer. It's good to know that it's normal.

Maybe, the xinput polling for"list of kbd_id of active keyboards" might be performed every 10 mins or 30 mins, then maybe the list is cleaned up by removing buttons or video bus from list, then listen to all "actual" keyboards.

For instance:

xinput | grep 'slave  keyboard' | sed -e's/slave *keyboard//' | grep keyboard | grep -o 'id=[0-9]*' | cut -d= -f2

decreases kbd_id list to 3 only.. Not sure if it will miss any "actual" keyboards though..

alperyilmaz commented 3 years ago

somehow, the following line finds keyboard ids as 5 11 and 17

xinput | grep 'slave  keyboard' | sed -e's/slave *keyboard//' | grep keyboard | grep -o 'id=[0-9]*' | cut -d= -f2

however the keyfreq.sh was collecting data from 5 and 17 only, missing the actual keyboard 11. So, I used the following line for filter out non-keyboard entries (taken from this post)

xinput --list | cut -d\[ -f1 | grep -i keyboard | egrep -iv 'virtual|video|button|bus' | egrep -o 'id=[0-9]+' | cut -f2 -d=

Naereen commented 3 years ago

Hi @alperyilmaz, Thanks for the suggestions. I no longer update uLogMe except for bug fixes, so I won't try these. But if you do try something and want to submit a pull-request, I will gladly look at it and merge it! Thanks in advance :pray:! I wish you a Happy New Year (in advance, or not, depending on your time zone)!

Naereen / uLogMe

many keyfreq.sh processes #35