search

2birds / Fledermaus

Use Ultraleap hand tracking to control a mouse
Apache License 2.0
6 stars 2 forks source link

macOS compatibility? #5

Open zachvalenti opened 3 months ago

zachvalenti commented 3 months ago

Thanks for making this project! I'm exploring using the Leap 2 as a pointer input with less risk for repetitive use injury as a podcast and video editor.

I managed to build and run this project on my 16" M1 Max laptop, but there was an immediate error reading the config file at the start. After successfully detecting my device, the app just idles, unresponsive to anything but "quit" with the message "Received unsupported message."

I wouldn't be surprised if I made a mistake somewhere (maybe I set up RapidJSON the wrong way), but I'm inexperienced enough that I don't know how to check. I'd appreciate any direction at all.

2birds commented 3 months ago

Hi Zach,

Well done for getting it built on Mac. As it stands, it won't work with Mac as I haven't implemented the cursor control commands for OSX. I will do that at some point, but I can't give a timescale, unfortunately. The config file error is probably another OSX-specific issue, I don't think it would even build without RapidJSON being in the right place. I expect it's an easy fix, but again, I really haven't tried very hard to make it work for Mac.

TL;DR, it's not you, it's me needing to do more 🙃

Stay tuned, and hopefully I'll have it working on Mac soon.

zachvalenti commented 3 months ago

Thank you! Tbh I wouldn't have gotten as far as I did without not one, but two LLM's suggesting next steps 😅

If I had more time I would keep going and attempt my first PR, but alas, I need to go back to the work I was hoping to be able to do by waving my hand around in the air 👋 Definitely will keep an eye here and give whatever version of Leap is available a try whenever it's ready for primetime.

Curious to know if the performance of your app is more usable than the included "Widgets" app "Pointer" demo? I found it too impercise for mousing around in REAPER (audio editing).

My sense is I'd probably see better results working out narrower, app-specific functions vs full system input control. REAPER specifically has a robust API/scripting system, but I'll need to wait until I have more free time to figure out plumbing between the sensor's output and *.lua scripts.

2birds commented 3 months ago

I'm absolutely delighted to have confounded an LLM 😁 If you really want to see them struggle, ask them to do cryptic crosswords.

I'm a former Ultraleap employee, and I created Fledermaus because Pointer didn't do quite what I wanted it to 😂 The Widgets were intended to be very lightweight, and allow you to compose a useful setup by using a subset of them. Fledermaus performs well on SIR170 and LMC2, not so much on the original LMC.

Interesting that you want it for REAPER. I use REAPER too, and built a VST3 plugin that uses Ultraleap hand tracking..

The problem you might have scripting a plugin for REAPER is that its scripting language is Lua. Ultraleap doesn't have a plugin for Lua, so you'd have to build that first. If you can get your hands on ultraleap-tracking-websocket (an unofficial revival of the old javascript server) you might be able to open a websocket with Lua and do something with the data being chucked over the LeapJS port.

Good luck, and keep an eye on Fledermaus, I'll sort OSX support..eventually