search

reynaldichernando / pinch

Simulate mouse input using hand tracking
https://www.chernando.com/blog/2023/07/23/hand-tracking-for-mouse-input.html
56 stars 6 forks source link

Infinite loading #1

Open tcsenpai opened 4 days ago

tcsenpai commented 4 days ago

Hello,

I managed to compile the project (I suggest adding apt install libxdo-dev for Ubuntu-derivatives as it fails otherwise).

This is what I get. The camera led does not even lights up.

image

System report:

@ fastfetch
             `..---+/---..`                 tcsenpai@neonova
         `---.``   ``   `.---.`             ----------------
      .--.`        ``        `-:-.          OS: KDE neon jammy 22.04 x86_64
    `:/:     `.----//----.`     :/-         Host: 82XM (IdeaPad Slim 3 15ABR8)
   .:.    `---`          `--.`    .:`       Kernel: Linux 6.10.11-x64v3-xanmod1
  .:`   `--`                .:-    `:.      Uptime: 3 days, 7 hours, 58 mins
 `/    `:.      `.-::-.`      -:`   `/`     Packages: 3730 (dpkg), 24 (flatpak), 30 (snap), 6 (brew)
 /.    /.     `:++++++++:`     .:    .:     Shell: python
`/    .:     `+++++++++++/      /`   `+`    Display (MSI MAG162V): 1920x1080 @ 60 Hz (as 1372x772) in 16″ [External]
/+`   --     .++++++++++++`     :.   .+:    Display (CMN1552): 1920x1080 @ 60 Hz (as 1601x900) in 16″ [Built-in] *
`/    .:     `+++++++++++/      /`   `+`    DE: KDE Plasma 6.1.4
 /`    /.     `:++++++++:`     .:    .:     WM: KWin (Wayland)
 ./    `:.      `.:::-.`      -:`   `/`     WM Theme: WillowDark
  .:`   `--`                .:-    `:.      Theme: Breeze (MoeDark) [Qt], Breeze-Dark [GTK2], Breeze [GTK3]
   .:.    `---`          `--.`    .:`       Icons: Fluent-dark [Qt], Fluent-dark [GTK2/3/4]
    `:/:     `.----//----.`     :/-         Font: Noto Sans (10pt) [Qt], Noto Sans (10pt) [GTK2/3/4]
      .-:.`        ``        `-:-.          Cursor: Future (24px)
         `---.``   ``   `.---.`             Terminal: WezTerm 20240203-110809-5046fc22
             `..---+/---..`                 Terminal Font: Agave Nerd Font
                                            CPU: AMD Ryzen 7 7730U (16) @ 4.55 GHz
                                            GPU: AMD Barcelo [Integrated]
                                            Memory: 7.67 GiB / 13.49 GiB (57%)
                                            Swap: 3.41 GiB / 8.00 GiB (43%)
                                            Disk (/): 237.08 GiB / 275.65 GiB (86%) - ext4
                                            Disk (/data): 311.53 GiB / 658.95 GiB (47%) - ext4
                                            Local IP (wlp2s0): 192.168.178.58/24
                                            Battery (L22B3PF2): 100% [AC Connected]
                                            Locale: it_IT.UTF-8
reynaldichernando commented 4 days ago

Hi @tcsenpai, thanks for opening up an issue I don't have an Ubuntu machine handy, so I couldn't test it right now

However, I can suggest checking the developer console in the pinch app (should be right click and then inspect), and see the error message there.

My suspicion is it failed to get the camera in this line https://github.com/reynaldichernando/pinch/blob/9e06b4c4635f05349a89d6122a7834ae8bed252c/src/down.js#L46

I did a quick research regarding the navigator.mediaDevices, and on macOS, there is an additional step for allowing the program to access the camera https://github.com/tauri-apps/tauri/discussions/5572 I suspect similar thing for Linux as well, but I haven't found any resource regarding that

reynaldichernando commented 4 days ago

If it is really a camera access issue, here is what I found:

Just did some digging around, and asked the Tauri team on their discord.

image

The main thing is Linux camera permission is not supported yet, due to webkitgtk not supporting webrtc.

However, I found another discussion on GitHub regarding the Linux camera support, and seems like this person did it, but it took some effort to build Tauri from source. https://github.com/tauri-apps/tauri/discussions/8426#discussioncomment-8268622

tcsenpai commented 4 days ago

And indeed:

image

Probably is something I could watch at but I am at the moment a bit overwhelmed by my projects. I'll resort writing something in Python if we cannot find a fix :)

(Yes, Python + MediaPipe should work better in this sense; but of course I don't expect you to rewrite the whole project in python. For inspiration, you can look at https://github.com/tcsenpai/mienmouse that I just made inspired by your project: it isn't nearly as polished but I wanted to experiment with a Pythonesque approach).