labstreaminglayer / LSL4Unity

A integration approach of the LabStreamingLayer Framework for Unity3D
Other
85 stars 39 forks source link

MarshalDirectiveException: cannot marshal type System.Single[,] for Android/Oculus #65

Open JustinHornsby opened 5 months ago

JustinHornsby commented 5 months ago

When built as an APK (Oculus Pro), the SimpleInletScaleObject.cs script throws an error: MarshalDirectiveException: cannot marshal type System.Single[,].

It works fine in the editor as well as PC VR, but standalone version will cause the issue. I've narrowed it down to SimpleInletScaleObject.cs, Line 35: uint res = dll.lsl_pull_chunk_f(inlet.obj, data_buffer, timestamp_buffer, (uint)data_buffer.Length, (uint)timestamp_buffer.Length, timeout, ref ec);

The same occurs for the other sample scenes provided by LSL's samples (as of 1.16.0)

I assume this may be more of an issue with Android, but I thought it would be good to bring it to your attention, just in case.

JustinHornsby commented 5 months ago

I have created a workaround. I essentially flatten the 2D array into a 1D array.

I changed SimpleInletScaleObject.cs, so that instead of it saying data_buffer = new float[buf_samples, n_channels]; it instead says: data_buffer_flat = new float[buf_samples * n_channels]; using a new float[] data_buffer_flat variable.

I then edited LSL.cs at line 640, (the pull_chunk variable that uses float[,])

public int pull_chunk(float[,] data_buffer, double[] timestamp_buffer, double timeout = 0.0)
        {
            int ec = 0;
            int numRows = data_buffer.GetLength(0);
            int numCols = data_buffer.GetLength(1);
            float[] data_buffer_flat = new float[numRows * numCols];

            for (int i = 0; i < numRows; i++)
            {
                for (int j = 0; j < numCols; j++)
                {
                    data_buffer_flat[i * numCols + j] = data_buffer[i, j];
                }
            }

            uint res = dll.lsl_pull_chunk_f(obj, data_buffer_flat, timestamp_buffer, (uint)data_buffer_flat.Length, (uint)timestamp_buffer.Length, timeout, ref ec);
            LSL.check_error(ec);
 // Unflatten the data_buffer_flat back into data_buffer
            for (int i = 0; i < numRows; i++)
            {
                for (int j = 0; j < numCols; j++)
                {
                    data_buffer[i, j] = data_buffer_flat[i * numCols + j];
                }
            }

            return (int)res / numCols;
        }

LSL.cs had the function "lsl_pull_chunk_f" edited as well at line 1115:

          [DllImport(libname, CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi, ExactSpelling = true)]
        public static extern uint lsl_pull_chunk_f(IntPtr obj, [In, Out] float[] data_buffer, [In, Out] double[] timestamp_buffer, uint data_buffer_elements, uint timestamp_buffer_elements, double timeout, ref int ec);

This has fixed the error with marshaling/sending data that uses [,] to Android/Oculus, while being sure to convert it back to a 2D array for Unity to handle.

This only works for floats[,] but doing it to the int[,] and double[,], etc should be similar.
It still works in the Editor and on Windows as well.

I uploaded as a .txt file if anyone wants to take a look at them. LSL.txt SimpleInletScaleObject.txt

cz2011301070 commented 4 months ago

I also had this problem. Snipaste_2024-07-05_12-04-01