search

ajayyy / Wiimote-FreePIE-VR-Controls

FreePIE plugin to try to make the Wiimote work as a SteamVR controller with a Leap Motion
32 stars 1 forks source link

Kinect + Wiimote instead of Leap Motion #7

Open BritishLad opened 6 years ago

BritishLad commented 6 years ago

sorry if i am late to this.

but im getting the errors i have attached below

errors

sorry about the second screen,

would love some help

thank you

ajayyy commented 6 years ago

Have you tried another file in the repo? I know some of them don't work for some people.

BritishLad commented 6 years ago

do you know where i would find one?

i was searching around and couldn't find anything

ajayyy commented 6 years ago

Inside my repository, there are other files you can try.

wiimote_leap_vive.py wiimote_leap_vive_2.py wiimote_leap_vive_nogyro.py

https://github.com/ajayyy/Wiimote-FreePIE-VR-Controls

Are you using leap motion?

BritishLad commented 6 years ago

no I'm not using leap motion

sorry i should have said.

ajayyy commented 6 years ago

ok, because the wiimote_vive.py also uses leap motion, that's probably why you are getting the error.

I will help in a few hours, gonna be busy sorry.

BritishLad commented 6 years ago

Its all good mate.

ill try the other files and remove the leap parts.

thanks for the help

Vininess commented 6 years ago

@BritishLad I get the same error on first time, just grab the latest version of freepie (1.10.692.0)

Some versions have some issues with DolphiiMote.dll, you can substitute this dll file too

BritishLad commented 6 years ago

@Vininess thx mate i substituted in a different DolphiiMote.dll and it worked

however Im not getting any feed back on the wiimotes

ajayyy commented 6 years ago

@BritishLad I would like to know what you are planning on doing, what are you going to use this with, how do you plan to do the positions of the hands, is it going to be the same as @Vininess ?

BritishLad commented 6 years ago

@ajayyy I will be doing the same as @Vininess yes, by using kinect, however if that doesnt work correctly as i hope i will use ps eye cameras and using different colour leds to track the wii motes

ajayyy commented 6 years ago

@BritishLad ok, so you guys just want button presses, I'll whip up a quick script for that for you to test and see if it works

ajayyy commented 6 years ago

@BritishLad and @Vininess Ok try wiimote_vive_only_buttons.py

BritishLad commented 6 years ago

@ajayyy yeah all works. thx for that.

however im not to sure if the kinect will work, as i cant find a tracker yet

ajayyy commented 6 years ago

@BritishLad I know you can use https://github.com/sheepsskullcity/opentrackkinect for positional tracking, but not controller positions.

And @Vininess was saying Driver4VR can do it

BritishLad commented 6 years ago

@ajayyy yes it can however it is A) a time limited program B) the tracking is sketchy

ajayyy commented 6 years ago

@BritishLad oh, ok. You can try opentrackkinect for positional tracking for now

ajayyy commented 6 years ago

@BritishLad have you tried https://github.com/jakers1403/Kinect-Plugin-Freepie

Edit: looks like I never got that working, but you could try it

BritishLad commented 6 years ago

I followed the steps to install it, however i think I'm having the same problem you did. as free pie wont open

ajayyy commented 6 years ago

aw that sucks

BritishLad commented 6 years ago

@ajayyy do u think there is a way to trick psmove into thinking the kinect is one of its webcams?

ajayyy commented 6 years ago

I remember them saying that they were going to add normal web cam support in the future, but they probably have not yet.

BritishLad commented 6 years ago

@ajayyy hey i got it working and however how would i be able to translate this into steamvr for tracking?

would i substitute it for the hydra positions?

ajayyy commented 6 years ago

what did you get working for positions?

BritishLad commented 6 years ago

@ajayyy i got tracking for the kinect to work, as well i used the acceleration of the wiimotes for the roll and pitch as i don't have motion plus. However how would i make it so the controllers stay in-front of my headset as the are stuck to one orientation not following the motion of the headset.

ill be busy for the next few hours 12 - 15 so ill come back on a few

BritishLad commented 6 years ago

@ajayyy any idea on how I would get it to stay with the headset?

ajayyy commented 6 years ago

That's what https://github.com/ajayyy/Wiimote-FreePIE-VR-Controls/blob/master/wiimote_leap_vive_2.py#L186 (line 186 and farther) does, you'd have to reverse engineer that and figure out how to that.

This assumes the rotation data can be pulled from freetrack. Are you using vridge, I'm not 100% sure if vridge puts the data to freetrack.

BritishLad commented 6 years ago

i have now made the hands stick to where my headset is positioned by using the kinect to track my head then taking it away from the hand positions, so my hands do move however i cant move the yaw of my hands.

as my wii motes dont have motions plus :/

any other way u know?

ajayyy commented 6 years ago

No, I think you would somehow need to track your hands with the kinect

BritishLad commented 6 years ago

am i better off getting the motion plus modules?

Vininess commented 6 years ago

@BritishLad with the wiimotion plus you got the yaw, because the motion plus adds more 3 giros to get a better respons... but you need to recalibrate every tome manually, because they don't have an magnetometer like in the psmove, the wii remote use the sensor bar to recalibrate

I try to figure out an idea to auto recalibrate the controller

Vininess commented 6 years ago

https://www.youtube.com/watch?v=jPP8XQRqlLA This video can be a solution, this guy create a code for less drifting

However i have two ideas 1: Make a code like in the video 2: buy a cheep magnetometer on aliexpress and soldering on wiiremote

ajayyy commented 6 years ago

That's a cool idea

Vininess commented 6 years ago

@ajayyy Do you mean the code or the soldering?

ajayyy commented 6 years ago

The soldering, I'm not sure how it would work but it would be cool. Maybe doing something like the original project this is forked from did https://github.com/fmaurer/FreePIE-VR-Controls

They made a custom controller out of an arduino.

Vininess commented 6 years ago

I have the tools and i know how to soldering, but i try to figure out where i have to solder the magnetometer to get the effect

BritishLad commented 6 years ago

@Vininess I have just ordered the motion plus adapters off of Amazon and i have completed/coded the kinect for my hand positions. When i have completed the code ill send @ajayyy and @Vininess a copy, if you wanna try it out.

Vininess commented 6 years ago

@BritishLad thanks for the help, i know a little how to code, but i know to soldering and math and physics stuffs, if you need some help in that parts cont on me

I bought the kinect today, but i discovered that i need an extension/power supply to plug on PC... so i back to the point 0

@BritishLad you coded the kinect using the freepie or another stuff? i heard/read, that the Driver4VR is quite good, but 20 euros converting to BRL, i will pay 80 BRL is quiet expensive... on kinect i pay 50 BRL, so the software is 2x the price, i want to try some alternatives

Thanks, when you send i check and feedback to you

BritishLad commented 6 years ago

first u will need this for ur FreePie https://github.com/sheepsskullcity/opentrackkinect this allows the plugin for FreePie to use the input/tracking.

The Code is a mix of @ajayyy wiimote controller buttons only and the kinect code.

def vive_controller1():

    finetuning = 23

    hydra[1].x = kinectPlugin.handLeftX*finetuning - kinectPlugin.headX
    hydra[1].y = kinectPlugin.handLeftY*finetuning - kinectPlugin.headY
    hydra[1].z = kinectPlugin.handLeftZ*30 - kinectPlugin.headZ

    hydra[1].pitch = -20

    Lone = wiimote[1].buttons.button_down(WiimoteButtons.DPadUp) or wiimote[1].buttons.button_down(WiimoteButtons.DPadDown) or wiimote[1].buttons.button_down(WiimoteButtons.DPadLeft) or wiimote[1].buttons.button_down(WiimoteButtons.DPadRight)
    Lfour = wiimote[1].buttons.button_down(WiimoteButtons.Plus)
    Lstart = wiimote[1].buttons.button_down(WiimoteButtons.Home)
    Lbumper = wiimote[1].buttons.button_down(WiimoteButtons.A)

    hydra[1].one = Lone
    hydra[1].four = Lfour
    hydra[1].start = Lstart
    hydra[1].bumper = Lbumper
    hydra[1].joybutton = Lone
    if wiimote[1].buttons.button_down(WiimoteButtons.DPadRight):
        hydra[1].joyx = 1
    elif wiimote[1].buttons.button_down(WiimoteButtons.DPadLeft):
        hydra[1].joyx = 0
    if wiimote[1].buttons.button_down(WiimoteButtons.DPadUp):
        hydra[1].joyy = 1
    elif wiimote[1].buttons.button_down(WiimoteButtons.DPadDown):
        hydra[1].joyy = 0

    hydra[0].x = kinectPlugin.handRightX*finetuning - kinectPlugin.headX
    hydra[0].y = kinectPlugin.handRightY*finetuning - kinectPlugin.headY
    hydra[0].z = kinectPlugin.handRightZ*30 - kinectPlugin.headZ

    hydra[0].pitch = -20

    Rone = wiimote[0].buttons.button_down(WiimoteButtons.DPadUp) or wiimote[0].buttons.button_down(WiimoteButtons.DPadDown) or wiimote[0].buttons.button_down(WiimoteButtons.DPadLeft) or wiimote[0].buttons.button_down(WiimoteButtons.DPadRight)
    Rfour = wiimote[0].buttons.button_down(WiimoteButtons.Plus)
    Rstart = wiimote[0].buttons.button_down(WiimoteButtons.Home)
    Rbumper = wiimote[0].buttons.button_down(WiimoteButtons.A)

    hydra[0].one = Rone
    hydra[0].four = Rfour
    hydra[0].start = Rstart
    hydra[0].bumper = Rbumper
    hydra[0].joybutton = Rone
    if wiimote[0].buttons.button_down(WiimoteButtons.DPadRight):
        hydra[0].joyx = 1
    elif wiimote[0].buttons.button_down(WiimoteButtons.DPadLeft):
        hydra[0].joyx = 0
    if wiimote[0].buttons.button_down(WiimoteButtons.DPadUp):
        hydra[0].joyy = 1
    elif wiimote[0].buttons.button_down(WiimoteButtons.DPadDown):
        hydra[0].joyy = 0

    joyLi = 0
    joyRi = 1

    diagnostics.watch(wiimote[joyLi].buttons.button_down(WiimoteButtons.B))
    diagnostics.watch(wiimote[joyRi].buttons.button_down(WiimoteButtons.B))

    diagnostics.watch(wiimote[joyLi].buttons.button_down(WiimoteButtons.A))
    diagnostics.watch(wiimote[joyRi].buttons.button_down(WiimoteButtons.A))
    diagnostics.watch(wiimote[joyRi].buttons.button_down(WiimoteButtons.DPadUp))
    diagnostics.watch(wiimote[joyLi].buttons.button_down(WiimoteButtons.DPadDown))

    diagnostics.watch(kinectPlugin.handLeftX)
    diagnostics.watch(kinectPlugin.handLeftY)
    diagnostics.watch(kinectPlugin.handLeftZ)

    diagnostics.watch(kinectPlugin.handRightX)
    diagnostics.watch(kinectPlugin.handRightY)
    diagnostics.watch(kinectPlugin.handRightZ)

def hydra_init():
    hydra[0].pitch = 0
    hydra[0].yaw = 0
    hydra[0].roll = 0
    hydra[0].x = 0
    hydra[0].y = 0
    hydra[0].z = 0

    hydra[1].pitch = 0
    hydra[1].yaw = 0
    hydra[1].roll = 0
    hydra[1].x = 0
    hydra[1].y = 0
    hydra[1].z = 0

def vive_init():
    global posx
    global posy
    global posz
    posx = 0
    posy = 0
    posz = 0
    global orrpitch
    global orryaw
    global orrroll
    orrpitch = 0
    orryaw = 0
    orrroll = 0

global s

if starting:
    vive_init()
    s = 1
    hydra_init()

if s == 1:
    vive_controller1()
Vininess commented 6 years ago

@BritishLad i will need the ping pong glowing to kinect tack?

BritishLad commented 6 years ago

No need for that, works without it

Vininess commented 6 years ago

@BritishLad i see, thanks, tomorrow i will try and feedback

ajayyy commented 6 years ago

@BritishLad Wow that's cool nice job. @Vininess aw that sucks you didn't get the pc adapter. I got mine with an Xbox so it already had it

ajayyy commented 6 years ago

What is kinectPlugin? Did you get the freepie plugin working?

BritishLad commented 6 years ago

@ajayyy I got it working with a new installation of freepie, I'll make a video of how I did it cause he didn't explain it very well.

BritishLad commented 6 years ago

The math for the hand positions is wrong at the moment thou. I just thought of it

''' Hydra[0].y = kinectPlugin.handLeftY kinectPlugin.headY ÷ 3 * 2 '''

Takes ur hight ÷ by 3 * 2 to find chest hight. Places hands there

Vininess commented 6 years ago

@BritishLad i try to run your code without the Kinect part, just for test, i use the Sixsense_Simple3D to test, i pull the trigger and nothing happens

Because i don't have the kinect cable yet i can't test the complete code

BritishLad commented 6 years ago

hmm, i think it might be the buttons code I'm gonna rewrite @ajayyy 's code for my own preference, cause i don't know the layout and then I'm gonna make a map of the buttons and upload it.

Vininess commented 6 years ago

Here is the layout of the Vive to Hydra

razer_hydra_button_mapping

I think that is the B for Trigger Dpad for TrackPad A for Press TrackPad Home For System

And i saw in other codes, minus for recalibrate

Vininess commented 6 years ago

@BritishLad and @ajayyy i didn't get the power supply for the kinect yet, but i have two questions

If i stand my back to the kinect, it can track my hands? assuming my body is between my hand and the sensor?

And the Kinect one is better than the Kinect 360 for freepie/VR purpose?

BritishLad commented 6 years ago

@Vininess you are correct with the sensor and you, as long as it can draw a skeleton around ur head shoulders and it will be able to track ur hands. and the kinect one is better due to its FOV, the one has a higher FOV. it also has better tracking, however kinect 360 does everything u need.