Streaming Point Clouds/ Long Throw depth data

[agroenenberg]

Hi, I have a problem with streaming the Long throw data to my PC. I keep getting the error ValueError: buffer size must be a multiple of element size whenever I run the python code for receiving both point cloud and depth sensor (long throw mode) on my PC.

This is the send function for the depth map:

    public void SaveLongThrowSensorDataEvent()
    {
#if ENABLE_WINMD_SUPPORT
        var LongdepthMap = researchMode.GetLongDepthMapBuffer();
#if WINDOWS_UWP
        if (tcpClient != null)
        {
            tcpClient.SendUINT16Async(LongdepthMap);
        }
#endif
#endif
    }

I tried changing the data = conn.recv(512*512*4+100) to data = conn.recv(288*320*4+100) because of the different size of the LT setting. Also, I do get data ( but cannot reshape it to any proper size) when using depth_img_np = np.frombuffer(data[5:5+N], np.uint8) instead of depth_img_np = np.frombuffer(data[5:5+N], np.uint16), but still, this is unusable. Does anyone know what I'm doing wrong? I am getting bytes from the socket but I can't get it into usable data...

[Mt-Perazim]

Hey, did you fix that problem? If yes, how?

[agroenenberg]

Hey, I got it working via the git from https://github.com/cgsaxner/HoloLens2-Unity-ResearchModeStreamer/ .

[Mt-Perazim]

oh, so you took another github repo for that. In that new repo, is there an option to get a point cloud like in this project? I'm getting frustrated with this one, because the Save-AHAT-Method does not work and the Save-Spatial-Image-Method throws this ValueError in python.... :/

--Edit: I created also an post about the things that dont work

[agroenenberg]

Maybe try and combine the two.

[Mt-Perazim]

@agroenenberg Hey, maybe I got the solution for your problem! I tried today to get a pointcloud for long- and short-depth-mode and both worked. How I achieved that for short, I posted in my last issue. But for the long-dept-point-cloud I used the following lines (code contains solution for short and long). And in your code above you used GetLongDepthMapBuffer():

    public void SavePointCloud()
    {
#if ENABLE_WINMD_SUPPORT
        var pointCloudBuffer = depthSensorMode == DepthSensorMode.ShortThrow
            ? researchMode.GetPointCloudBuffer()
            : researchMode.GetLongThrowPointCloudBuffer();
#if WINDOWS_UWP
        if (tcpClient != null)
            tcpClient.SendSingleAsync(pointCloudBuffer);
#endif
#endif
    }

[agroenenberg]

Thanks for the solution! It's great to have a backup for if the other method turns out to be insufficient!