search

Nat-Lab / chunithm-touch

Touchscreen and Leap Motion support for Chunithm.
The Unlicense
39 stars 1 forks source link

Leap Motion? #1

Open hahaluckyme opened 4 years ago

hahaluckyme commented 4 years ago

Hi, are you are planning for Leap Motion? I saw it in your description but not in the code.

magicnat commented 4 years ago

You can find it under the "leap" branch: https://github.com/Nat-Lab/chunithm-touch/tree/leap.

Here's a build of the latest commit from the leap branch: chuni-touch-leap.zip.

hahaluckyme commented 4 years ago

Thank you! Amazing. Have you used Brokenithm? I think using Brokenithm for the LED controller and Leap Motion for IR sensor would be a truly powerful combination.

hahaluckyme commented 4 years ago

I was able to get it working, and it works very well. Maybe the IR images can be used to sense depth instead of doing hand tracking, because crossing hands over each other seems to lose tracking very easily causing misses.

magicnat commented 4 years ago

Thank you! Amazing. Have you used Brokenithm? I think using Brokenithm for the LED controller and Leap Motion for IR sensor would be a truly powerful combination.

I used it before - it has some noticeable input latency, so I didn't use it that much, and here's another project that I worked on before that has LED output: https://github.com/Nat-Lab/chunithm-vcontroller. It has some input latency tho. That's why I started this project.

Maybe the IR images can be used to sense depth instead of doing hand tracking, because crossing hands over each other seems to lose tracking very easily causing misses.

I noticed that too. Unfortunately, there's no API from leap for depth tracking. It does allow you to get raw image pixels from both IR cameras. I imagine with some CV we might be able to get depth from the images, but I'm not familiar with those.

Alternately try to use the leap device head-mounted. It seems to track crossing hands better when it is head-mounted.

hahaluckyme commented 4 years ago

I tried head mounted but I was getting worse tracking than from below. My current best solution is to place it under the game surface a few inches and almost back at where the elbows are. Is there some special setting for head tracking that I have to enable or are you just getting decent tracking by default?

Also, I was able to disable much of the crappy Windows touchscreen input gestures, but I still get slide misses sometimes. Have you figured out a way to disable all the gesture junk? I am using a Cintiq Pro 13.

If you have Discord, feel free to add me at hahaluckyme#0001!

hahaluckyme commented 4 years ago

I used it before - it has some noticeable input latency, so I didn't use it that much, and here's another project that I worked on before that has LED output: https://github.com/Nat-Lab/chunithm-vcontroller. It has some input latency tho. That's why I started this project.

This was very interesting, the separated keyboard in another app is actually preferable in my case, since the keyboard can be rendered on a Cintiq pro separately from the game app. I think Leap + chunithm-vcontroller is an ideal situation for me, if I can merge the inputs.

magicnat commented 4 years ago

I tried head mounted but I was getting worse tracking than from below.

The head-mounted mode did help with the hand-crossing issue for me. It works best when I stand up and look down at the touchscreen - which is a very weird way to play. I was thinking of mounting the leap device somewhere and have it points down to the screen, but I didn't have the time and equipment to try that.

My current best solution is to place it under the game surface a few inches and almost back at where the elbows are.

We got a similar setup - the setup that works best for me looks like this.

there some special setting for head tracking that I have to enable or are you just getting decent tracking by default?

It works pretty well out of the box for me - as long as my hands are in its view all the time and they are not crossing. I think there's a diagnostic visualizer included in the leap motion control panel somewhere - maybe try that and see what works best?

I was able to disable much of the crappy Windows touchscreen input gestures, but I still get slide misses sometimes.

Hmm. I'm on Windows 10 and I haven't noticed anything weird. Maybe the ir_keep_slider option could help? Some people are reporting their slides being registered as air-action.

The separated keyboard in another app is actually preferable in my case.

I have planned for implementing a mode where a separate window will be open for touch input, but I am too busy dealing with stuffs and haven't got time to do any of that. Plus Leap Motion not working well really defeats my motivation.

If you have Discord, feel free to add me at hahaluckyme#0001!

I don't really use discord, sorry.

magicnat commented 4 years ago

In case you are still interested, I added support for controlling on a separate window in version 0.4.0.

Add the following to chunitouch.ini to enable:

[options]
separate_control = 1
hahaluckyme commented 4 years ago

Thank you!! :)

My friend bought a Chunithm controller so I've been playing on that a lot. Still a long way to go... D:

joannawolthuis commented 1 year ago

Hi :) @hahaluckyme This is years late, but just in case you see it I'd be really interested in how you combined brokenithm with leap motion. New to the game and trying to get a good enough starting setup!