search

MyRobotLab / InMoov

inmoov repo
http://myrobotlab.org
95 stars 66 forks source link

webcam + kinect at same time #96

Open hairygael opened 7 years ago

hairygael commented 7 years ago

Hello Anthony, Some enhancements ideas, in case you get bored. :) Do you think we could do something about the problem of having to select one or the other but not both at the same time? Currently if the kinect is connected, the webcam gets an error with OpenCv. I also think it would help if we could define if the head has two or one webcam. Nice when doing stereoscopic. It's also useful if we want to play with the Oculus rift.

moz4r commented 7 years ago

thanks for filling unused seconds, because of inmoov never bored anymore :) About what I know we can use 2 different wecam but not the same id, inside windows Os. As soon I can I will test this.

image

hairygael commented 7 years ago

Yeah! Ouch! Rachel is having a big summer lifting.

Gael Langevin Creator of InMoov InMoov Robot http://www.inmoov.fr @inmoov http://twitter.com/inmoov

2017-07-28 0:18 GMT+02:00 Anthony notifications@github.com:

thanks for filling unused seconds, because of inmoov never bored anymore :) About what I know we can use 2 different wecam but not the same id, inside windows Os. As soon I can I will test this.

[image: image] https://user-images.githubusercontent.com/18124594/28694501-23bbb166-732a-11e7-83b4-1efe1a291e50.png

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MyRobotLab/inmoov/issues/96#issuecomment-318502488, or mute the thread https://github.com/notifications/unsubscribe-auth/AIF2x43Apguo6QJQ-AUVnrtWxJ-QFbIBks5sSQzHgaJpZM4OjvX2 .

supertick commented 7 years ago

A couple notes :

hairygael commented 7 years ago

Thanks Grog for the information! How does window work out the different cameras? Can they be specified in the device manager, a little bit like com ports? Can you give them an entry number or key?

Gael Langevin Creator of InMoov InMoov Robot http://www.inmoov.fr @inmoov http://twitter.com/inmoov

2017-07-28 22:05 GMT+02:00 GroG notifications@github.com:

A couple notes :

  • The refactored version of OpenCV (called Vision) uses input as a filter, so multiple can be added to the same pipeline
  • In the future the general idea is to remove all the "hidden" creation of services - and scripts will be responsible for creating all the services/blocks they want to play with, so creating 2 OpenCVs or other vision services is trivial - we don't want to do this before the release, but we will be moving towards this design in the future. The only "magical" part of composite services are in the "attach" method not the starting ..

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MyRobotLab/inmoov/issues/96#issuecomment-318749456, or mute the thread https://github.com/notifications/unsubscribe-auth/AIF2x_mYOrvF8L5Py_uibLPBgZ_2dvt9ks5sSj8DgaJpZM4OjvX2 .

ghost commented 7 years ago

When you connect ther usb to the computer it is automatically assigned a port number of 0 - MAXUSB ports, Similar to how you would connect Arduinos to COM. So if you have say three cameras they will be assigned in an order of which was pluged in first

On linux you can scan your usb ports with lsusb

lsusb

Now when you implement this you are simply opening two new source streams from opencv.

edit:

# By defualt 0 is your built in webcam
# Then after that you have the three other cameras Wich should be 1, 2, 3 
# Or it could be 0, 1, 2 if you dont have a webcam
# But dont take my word for it.
LeftEye = cv2.VideoCapture(1)
RightEye = cv2.VideoCapture(2)
Kinect = cv2.VideoCapture(3)

    # Create multiple filters
    Left_ret, Left_frame = LeftEye.read()
    Right_ret, Right_frame = RightEye.read()
    K_ret, K_frame = Kinect.read()

    # Masks from here on

@hairygael

hairygael commented 7 years ago

Thanks Josh, It is pretty much what I assumed, but why can't I make the webcam work and the kinect work at the same time with Windows? I am not even trying with MRL, but just in Windows. For some reason only one of the two is detected, the other doesn't even mount. If the webcam is first connected and detected when I then connect the kinect, the webcam disappear.

Gael Langevin Creator of InMoov InMoov Robot http://www.inmoov.fr @inmoov http://twitter.com/inmoov

2017-07-30 16:14 GMT+02:00 Josh Brown notifications@github.com:

When you connect ther usb to the computer it is automatically assigned a port number of 0 - MAXUSB ports, Similar to how you would connect Arduinos to COM. So if you have say three cameras they will be assigned in an order of which was pluged in first

On linux you can scan your usb ports with lsusb

[image: lsusb] https://user-images.githubusercontent.com/17482298/28754226-4baa7c66-750f-11e7-8744-2942d2b87fc3.png

Now when you implement this you are simply opening two new source streams from opencv.

Var LeftEye = 1 Var RightEye = 2

@hairygael https://github.com/hairygael

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/MyRobotLab/inmoov/issues/96#issuecomment-318904549, or mute the thread https://github.com/notifications/unsubscribe-auth/AIF2x2uNqQLRX8OnJyVGO_5YfjnieGboks5sTI-0gaJpZM4OjvX2 .

ghost commented 7 years ago

From what i understand, inorder to use opencv with the kinect you need a special library for the depth sensors. as well as you will need:

cython
python-dev
python-numpy
# Do this last as the above are dependecies
freenect

Note, if you have the kinect | v2 then you will need libfreenect2

Code Example For Just Kinect

#import the necessary modules
import freenect
import cv2
import numpy as np

#function to get RGB image from kinect
def get_video():
    array,_ = freenect.sync_get_video()
    array = cv2.cvtColor(array,cv2.COLOR_RGB2BGR)
    return array

#function to get depth image from kinect
def get_depth():
    array,_ = freenect.sync_get_depth()
    array = array.astype(np.uint8)
    return array

if __name__ == "__main__":
    while 1:
        #get a frame from RGB camera
        frame = get_video()
        #get a frame from depth sensor
        depth = get_depth()
        #display RGB image
        cv2.imshow('RGB image',frame)
        #display depth image
        cv2.imshow('Depth image',depth)

        # quit program when 'esc' key is pressed
        k = cv2.waitKey(5) & 0xFF
        if k == 27:
            break
    cv2.destroyAllWindows()

Then just execute the program like normal python filename.py

Resources

I dont exactly have a kinect in my hands currently so i wont be able to give you much help. But i did some googling and came up with this @hairygael

kwatters commented 7 years ago

A few comments about getting the kinect and a webcam to work at the same time.

  1. the kinect definitely needs it's external power supply. (part of the normal kinect power cables.)
  2. the kinect takes a lot of bandwidth on a usb 2.0 and sometimes that conflicts with the webcam.
  3. when a webcam turns on, it draws extra power ...
  4. running a kinect or a webcam through a usb hub almost never works as expected.

that being said.. it can be made to work. you need to make sure your usb ports have enough power and that , if possible, the kinect and the webcam are on different usb hubs/ports on the computer so they don't compete with each other.

long long ago, i was able to finally get the right wiring down that I was able to start 2 webcams and 1 kinect, http://www.myrobotlab.org/content/2-cameras-1-kinect

hairygael commented 7 years ago

@kwatters , Yes that is the issue. InMoov is running on the Lenovo Tablet under Windows 8.1 which has only one mini USB 3 port. The same USB port is also powering the tablet and having a 8 port hub. On the hub, I have the two Arduino Mega, the camera, the Kinect, the Bluetooth dongle for the headset, and the USB sound card. Too much competition at the entrance of the port...