Filtered depth map to frame

AAcquier commented 5 years ago

I am extracting the depth map from pylibfreenect2 suing the asarray function as I want to filter it to eliminate noise and 'holes' before the registration.

Is there a way to transform back this array of filtered depth into a frame to do the registration?
If it is possible what are the steps to be taken?

Hoping to hear from you soon

Kind regards,

Alex

r9y9 commented 5 years ago

You can pass your numpy array to construct a Frame object. See http://r9y9.github.io/pylibfreenect2/latest/api.html#pylibfreenect2.libfreenect2.Frame and https://github.com/r9y9/pylibfreenect2/blob/4bea7cb58798dec75585768cf5a8fb4084813cd1/pylibfreenect2/libfreenect2.pyx#L220-L223.

r9y9 commented 5 years ago

Code would look like frame = Frame(512, 424, 4, your_array)

AAcquier commented 5 years ago

Using the line frame = Frame(512, 424, 4, your_array) It returns the error: File "kinectDataViewer.py", line 75, in <module> new_depth = Frame(512, 424,2, depth_test) File "pylibfreenect2/libfreenect2.pyx", line 231, in pylibfreenect2.libfreenect2.Frame.__cinit__ TypeError: only size-1 arrays can be converted to Python scalars Using the untouched data from the depth frame extracted using the asarray() method. The raw output of the array is sown below: [[ 0. 0. 2124.8354 ... 810.58075 827.82336 0. ] [ 0. 1969.0292 0. ... 827.20984 825.15436 0. ] [ 0. 0. 0. ... 827.20056 824.5674 0. ] ... [ 0. 0. 0. ... 0. 0. 0. ] [ 0. 0. 0. ... 0. 0. 0. ] [ 0. 0. 0. ... 0. 0. 0. ]]

r9y9 commented 5 years ago

https://github.com/r9y9/pylibfreenect2/blob/4bea7cb58798dec75585768cf5a8fb4084813cd1/pylibfreenect2/libfreenect2.pyx#L235-L236

As seen in the code, numpy array shoud be 2-dim and of type float32. And bytes_per_pixel should be 4 (float32). Please check your input.

AAcquier commented 5 years ago

Now I get this error message File "kinectDataViewer.py", line 76, in <module> new_depth = Frame(512, 424,4, [depth_test]) File "pylibfreenect2/libfreenect2.pyx", line 231, in pylibfreenect2.libfreenect2.Frame.__cinit__ TypeError: an integer is required

r9y9 commented 5 years ago

I forgot the frame_type argment.

Frame(512, 424,4, FrameType.Depth, depth_test)

AAcquier commented 5 years ago

cinit takes only 4 arguments: File "kinectDataViewer.py", line 76, in <module> new_depth = Frame(512, 424,4, FrameType.Depth,depth_test) File "pylibfreenect2/libfreenect2.pyx", line 230, in pylibfreenect2.libfreenect2.Frame.__cinit__ TypeError: __cinit__() takes at most 4 positional arguments (5 given)

ozgraslan commented 5 years ago

I have the same problem. Any updates about this issue?

ozgraslan commented 5 years ago

Using the latest version I was able to record Frames as numpy arrays.

AAcquier commented 5 years ago

The problem is how to transform these numpy arrays back into frames

On Tue, 30 Jul 2019, 08:48 JAKD9, notifications@github.com wrote:

Using the latest version I was able to record Frames as numpy arrays.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/r9y9/pylibfreenect2/issues/63?email_source=notifications&email_token=AF3T5XYQJXFTHRFUFKYCKCTQB7W3RA5CNFSM4HS3DF62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3DDKUA#issuecomment-516306256, or mute the thread https://github.com/notifications/unsubscribe-auth/AF3T5X5DVE4TG6XL64ONJW3QB7W3RANCNFSM4HS3DF6Q .

ozgraslan commented 5 years ago

Using Frame(512, 424,4, FrameType.Depth, depth_test) worked for me with the latest version. With the stable version I got the same error you got.

AAcquier commented 5 years ago

Okay I gotta give another look to that as this did not work for me but I wont be able to look at it for a while as I have other other projects

http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Virus-free. www.avg.com http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Tue, Jul 30, 2019 at 9:30 AM JAKD9 notifications@github.com wrote:

Using Frame(512, 424,4, FrameType.Depth, depth_test) worked for me.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/r9y9/pylibfreenect2/issues/63?email_source=notifications&email_token=AF3T5XYX6AVMOLUTXBSJOSDQB7323A5CNFSM4HS3DF62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3DG42Y#issuecomment-516320875, or mute the thread https://github.com/notifications/unsubscribe-auth/AF3T5X62VUKPVTBQXMJ4NETQB7323ANCNFSM4HS3DF6Q .

r9y9 commented 5 years ago

Oh, oops, I didn't update pypi version for a while. Thanks for pointing out the issue, @JAKD9. I will release a new version soon.

Diankuang-Wu commented 4 years ago

When I run the code below, it appears error: Traceback (most recent call last): File "Dispaly_multiframes.py", line 82, in color_depth_map=color_depth_map) File "pylibfreenect2/libfreenect2.pyx", line 1132, in pylibfreenect2.libfreenect2.Registration.apply AssertionError

frames = listener.waitForNewFrame()
    color = frames["color"]
    ir = frames["ir"]
    depth = frames["depth"]

new_depth = Frame(512, 424, 4, 4, numpy_array=depth.asarray())

registration.apply(color, new_depth, undistorted, registered,
                       bigdepth=bigdepth,
                       color_depth_map=color_depth_map)

the error appears when I want to get a registered image, and it seems that the attribute of depth.take_ownership should equal to False in libfreenect2.pyx line 1132. However , the former of new_depth = Frame(512, 424, 4, 4, numpy_array=depth.asarray()) causes the depth.take_ownership =True in libfreenect2.pyx line 245, how could I solve this problem?

Hoping to hear from you soon

Kind regards,

Bin Chen

r9y9 / pylibfreenect2

Filtered depth map to frame #63