Franzisdrak commented 5 years ago

Hi there,

i am just getting started with opencv and pykinect2.

The examples with pygame work for me. But i can't seem to get the img read with opencv to work. Do you have any tips on how to access the img stream with opencv? I need the image Manipulation i can do with it.

Some Info about my system: Windows 10 Kinect SDK v2.0 python 3.6.5 Anaconda OpenCV 3.4.3

Kind regards Franzisdrak

Franzisdrak commented 5 years ago

Got it working. Now my image is super noisy. I am probably missing sth.

from pykinect2 import PyKinectV2
from pykinect2.PyKinectV2 import *
from pykinect2 import PyKinectRuntime
import numpy as np
import cv2

kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Infrared)

while True:
    # --- Getting frames and drawing
    if kinect.has_new_infrared_frame():
        frame = kinect.get_last_infrared_frame()
        frame = frame.astype(np.uint8)
        frame = np.reshape(frame, (424, 512))
        img = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
        output = cv2.bilateralFilter(img, 9, 150, 75)
        cv2.imshow('KINECT Video Stream', output)
        frame = None

    key = cv2.waitKey(1)
    if key == 27: break

fixed it! The data is not of type uint8 but 16. Therefore my image was not correctly displayed.

yxp1003 commented 5 years ago

搞定了。现在我的形象超级嘈杂。我可能错过了......

from pykinect2 import PyKinectV2
from pykinect2.PyKinectV2 import *
from pykinect2 import PyKinectRuntime
import numpy as np
import cv2

kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Infrared)

while True:
    # --- Getting frames and drawing
    if kinect.has_new_infrared_frame():
        frame = kinect.get_last_infrared_frame()
        frame = frame.astype(np.uint8)
        frame = np.reshape(frame, (424, 512))
        img = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
        output = cv2.bilateralFilter(img, 9, 150, 75)
        cv2.imshow('KINECT Video Stream', output)
        frame = None

    key = cv2.waitKey(1)
    if key == 27: break

固定它！数据不是类型uint8而是16.因此我的图像没有正确显示。

hello, How do you fixed it, I run it and didn't show anything

nitzel commented 5 years ago

I guess by using uint16 instead of uint8 in frame = frame.astype(np.uint8), but @Franzisdrak will be able to elaborate ... By the way, examples do exist: https://github.com/Kinect/PyKinect2/blob/master/examples/PyKinectInfraRed.py

yxp1003 commented 5 years ago

I have tried it，but still does't works，have u succeed show your video in the window？

yxp1003 commented 5 years ago

I guess by using uint16 instead of uint8 in frame = frame.astype(np.uint8), but @Franzisdrak will be able to elaborate ... By the way, examples do exist: https://github.com/Kinect/PyKinect2/blob/master/examples/PyKinectInfraRed.py

Still failed, Exactly, I wan't to process the video capture by OpenCV, so I don't sure that the Variable format in the example is suitable.

yxp1003 commented 5 years ago

Got it working. Now my image is super noisy. I am probably missing sth.

from pykinect2 import PyKinectV2
from pykinect2.PyKinectV2 import *
from pykinect2 import PyKinectRuntime
import numpy as np
import cv2

kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Infrared)

while True:
    # --- Getting frames and drawing
    if kinect.has_new_infrared_frame():
        frame = kinect.get_last_infrared_frame()
        frame = frame.astype(np.uint8)
        frame = np.reshape(frame, (424, 512))
        img = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
        output = cv2.bilateralFilter(img, 9, 150, 75)
        cv2.imshow('KINECT Video Stream', output)
        frame = None

    key = cv2.waitKey(1)
    if key == 27: break

fixed it! The data is not of type uint8 but 16. Therefore my image was not correctly displayed.

I check the excample "PyKinectBodyGame.py", and found that its type is uint8, plz tell me how did u solved problem finally, thx.

PanJinquan commented 4 years ago

我参考PyKinectInfraRed.py，写了一版使用OpenCV显示的方法，跟PyKinectInfraRed.py效果差钱不多： `python

--coding: utf-8 --

""" @Project: PyKinect2-OpenCV @File : demo01.py @Author : panjq @E-mail : pan_jinquan@163.com @Date : 2019-10-07 11:16:05 """ from pykinect2 import PyKinectV2 from pykinect2 import PyKinectRuntime import numpy as np import cv2

kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Infrared) depth_width, depth_height = kinect.depth_frame_desc.Width, kinect.depth_frame_desc.Height # Default: 512, 424

while True:

--- Getting frames and drawing

if kinect.has_new_infrared_frame():
    frame = kinect.get_last_infrared_frame()
    frame = frame.astype(np.uint16)              # infrared frame是uint16类型
    frame = np.uint8(frame.clip(1, 4080) / 16.)  # 转换为uint8时，需要避免溢出255*16=4080
    frame = np.reshape(frame, (depth_height, depth_width))
    frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
    cv2.imshow('KINECT Video Stream', frame)
    frame = None

key = cv2.waitKey(1)
if key == 27:
    break

`

hayatkhan8660-maker commented 2 years ago

Got it working. Now my image is super noisy. I am probably missing sth.

from pykinect2 import PyKinectV2
from pykinect2.PyKinectV2 import *
from pykinect2 import PyKinectRuntime
import numpy as np
import cv2

kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Infrared)

while True:
    # --- Getting frames and drawing
    if kinect.has_new_infrared_frame():
        frame = kinect.get_last_infrared_frame()
        frame = frame.astype(np.uint8)
        frame = np.reshape(frame, (424, 512))
        img = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
        output = cv2.bilateralFilter(img, 9, 150, 75)
        cv2.imshow('KINECT Video Stream', output)
        frame = None

    key = cv2.waitKey(1)
    if key == 27: break

fixed it! The data is not of type uint8 but 16. Therefore my image was not correctly displayed.

I am trying with the same approach, but still doesn't work!!!!
Here is my script

kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Infrared)

while True:

--- Getting frames and drawing

if kinect.has_new_infrared_frame():
    frame = kinect.get_last_infrared_frame()
    frame = frame.astype(np.uint16)
    frame = np.reshape(frame, (424, 512))
    img = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
    output = cv2.bilateralFilter(img, 9, 150, 75)
    cv2.imshow('KINECT Video Stream', output)
    frame = None

if cv2.waitKey(1) & 0xFF == ord('q'):
    break

KonstantinosAng commented 2 years ago

try this

from pykinect2 import PyKinectV2 from pykinect2 import PyKinectRuntime import numpy as np import cv2

kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Infrared) depth_width, depth_height = kinect.depth_frame_desc.Width, kinect.depth_frame_desc.Height # Default: 512, 424

while True:

--- Getting frames and drawing

if kinect.has_new_infrared_frame(): frame = kinect.get_last_infrared_frame() frame = frame.astype(np.uint16) # infrared frame是uint16类型 frame = np.uint8(frame.clip(1, 4080) / 16.) # 转换为uint8时，需要避免溢出255*16=4080 frame = np.reshape(frame, (depth_height, depth_width)) frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR) frame = cv2.bilateralFilter(frame, 9, 150, 75) cv2.imshow('KINECT Video Stream', frame) frame = None

key = cv2.waitKey(1) if key == 27: break

hayatkhan8660-maker commented 2 years ago

try this

from pykinect2 import PyKinectV2 from pykinect2 import PyKinectRuntime import numpy as np import cv2

kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Infrared) depth_width, depth_height = kinect.depth_frame_desc.Width, kinect.depth_frame_desc.Height # Default: 512, 424

while True:

--- Getting frames and drawing

if kinect.has_new_infrared_frame(): frame = kinect.get_last_infrared_frame() frame = frame.astype(np.uint16) # infrared frame是uint16类型 frame = np.uint8(frame.clip(1, 4080) / 16.) # 转换为uint8时，需要避免溢出255*16=4080 frame = np.reshape(frame, (depth_height, depth_width)) frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR) frame = cv2.bilateralFilter(frame, 9, 150, 75) cv2.imshow('KINECT Video Stream', frame) frame = None

key = cv2.waitKey(1) if key == 27: break

Run without error, but nothing to display? same as previous.

KonstantinosAng commented 2 years ago

try without the bilateral

from pykinect2 import PyKinectV2 from pykinect2 import PyKinectRuntime import numpy as np import cv2

kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Color | PyKinectV2.FrameSourceTypes_Infrared) depth_width, depth_height = kinect.depth_frame_desc.Width, kinect.depth_frame_desc.Height # Default: 512, 424

while True:

if kinect.has_new_infrared_frame(): frame = kinect.get_last_infrared_frame() frame = frame.reshape((kinect.infrared_frame_desc.Height, kinect.infrared_frame_desc.Width)).astype(np.uint16)

frame = cv2.bilateralFilter(frame, 9, 150, 75)

cv2.imshow('KINECT Video Stream', frame)
frame = None

key = cv2.waitKey(1) if key == 27: break

hayatkhan8660-maker commented 2 years ago

frame = frame.reshape((kinect.infrared_frame_desc.Height, kinect.infrared_frame_desc.Width)).astype(np.uint16)

frame = cv2.bilateralFilter(frame, 9, 150, 75)

============ Still doesnt work :(

Here is the updated script. from pykinect2 import PyKinectV2 from pykinect2 import PyKinectRuntime import numpy as np import cv2

kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Infrared) depth_width, depth_height = kinect.depth_frame_desc.Width, kinect.depth_frame_desc.Height # Default: 512, 424

while True: if kinect.has_new_infrared_frame(): frame = kinect.get_last_infrared_frame() frame = frame.reshape((kinect.infrared_frame_desc.Height, kinect.infrared_frame_desc.Width)).astype(np.uint16)

frame = cv2.bilateralFilter(frame, 9, 150, 75)

    cv2.imshow('KINECT Video Stream', frame)
    frame = None

key = cv2.waitKey(1)
if key == 27:
    break

KonstantinosAng commented 2 years ago

Are you able to display any frame from kinect, color, depth, infrared or body?

hayatkhan8660-maker commented 2 years ago

Are you able to display any frame from kinect, color, depth, infrared or body?

yes! the color and depth frames are displayed but I need IR image for projection mapping. I have tried with different settings but unable to make it......

Did you display the IR frame?

KonstantinosAng commented 2 years ago

Yes I did.

Have you tried this example:

https://github.com/Kinect/PyKinect2/blob/master/examples/PyKinectInfraRed.py

hayatkhan8660-maker commented 2 years ago

Thanks, @KonstantinosAng I solve it... The main problem was the older version of pykinect2 libraries, I simply replace it with the latest version...

Really appreciate your cooperation...

JBelmontte commented 2 years ago

Hi! Finally I could use my Kinect v2 as a camera (Color) using OpenCV using your code:

from pykinect2 import PyKinectV2 from pykinect2 import PyKinectRuntime import numpy as np import cv2

kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Color | PyKinectV2.FrameSourceTypes_Depth)

while True: if kinect.has_new_color_frame(): frame = kinect.get_last_color_frame() frame = frame.reshape((1080, 1920, 4)) cv2.imshow('KINECT Video Stream', frame) frame = None key = cv2.waitKey(1) if key == 27: break

Now I'm trying to use de depth sensors to know the distance from an object in a (x,y) position to the camera. Can I use for example x, y = 960, 540 (and draw it on the frame) and see the depth to that point? Thanks

KonstantinosAng commented 2 years ago

You have to be careful in what coordinate system you are. If the x, y is on the Color Cordinate System then you can paint it on the color image but in order to find its depth you have to use the Depth Cordinate System. That means you have to get the color point to the depth Space:

Using the mapper.py in my repo

from pykinect2.PyKinectV2 import * from pykinect2 import PyKinectV2 from pykinect2 import PyKinectRuntime import mapper

kinect = PyKinectRuntime.PyKinectRuntime(PyKinectV2.FrameSourceTypes_Color | PyKinectV2.FrameSourceTypes_Body | PyKinectV2.FrameSourceTypes_Depth)

depth_x, depth_y = mapper.color_point_2_depth_point(kinect, _DepthSpacePoint, kinect._depth_frame_data, [960, 540]) # pixel

""" Then you can use these depth pixel points to get the actual depth from the kinect sensor """

depth = kinect.get_last_depth_frame()

"""--------------------------------------------------------------------------------------- There is no guaranty that a depth value will have a corresponding depth pixel ---------------------------------------------------------------------------------------""" if (int(depth_y ) 512 + int(depth_x)) < 512 424: depth_z = float(depth[int(depth_y ) * 512 + int(depth_y )]) # mm else:

If it exceeds return the last value to catch overflow

  depth_z = float(depth[int((512 * 424) - 1)])  # mm

JBelmontte commented 2 years ago

Many thanks for your quick response! Awesome wrapper, super useful for my purpose.

Kinect / PyKinect2

Using Opencv to read VideoStream #61

--coding: utf-8 --

--- Getting frames and drawing

I am trying with the same approach, but still doesn't work!!!!
Here is my script

--- Getting frames and drawing

--- Getting frames and drawing

--- Getting frames and drawing

frame = cv2.bilateralFilter(frame, 9, 150, 75)

frame = cv2.bilateralFilter(frame, 9, 150, 75)

============ Still doesnt work :(

frame = cv2.bilateralFilter(frame, 9, 150, 75)

Using the mapper.py in my repo

If it exceeds return the last value to catch overflow

Kinect / PyKinect2

Using Opencv to read VideoStream #61

--coding: utf-8 --

--- Getting frames and drawing

I am trying with the same approach, but still doesn't work!!!! Here is my script

--- Getting frames and drawing

--- Getting frames and drawing

--- Getting frames and drawing

frame = cv2.bilateralFilter(frame, 9, 150, 75)

frame = cv2.bilateralFilter(frame, 9, 150, 75)

============ Still doesnt work :(

frame = cv2.bilateralFilter(frame, 9, 150, 75)

Using the mapper.py in my repo

If it exceeds return the last value to catch overflow

I am trying with the same approach, but still doesn't work!!!!
Here is my script