Memory Issue In Buffers

[omaralcheikh]

Hello,

I have a large array that needs to be sent over a socket connection from a server (Node Js) to a Client (Holo Js / Hololens) every second or so. Sending it as a normal array is kind of slow so i resorted to converting the list to a buffer.

var buffer = new Float64Array(list).buffer
var data = {
    buffer : buffer
}
client.emit('sendData', data);

However, I realized that the app would crash and start throwing an out of memory exception after a while. After looking into it more i realized that the memory displayed on the diagnostic tools view in visual studio keeps increasing every time a new buffer is being received. This does not happen when receiving a normal array.

I attached a sample node js server NodeJs - Socket Server.zip

And here is a small script to run on HoloJs to replicate the problem.

var url = "http://{ip}:{port}/";
var socket = io(url, { transports: ['websocket'] });

socket.on('connect', function () {
    console.log('connected');
    socket.emit('requestData');
});

socket.on('sendData', function (data) {
    socket.emit('requestData');
});

Another thing i tried that caused the same behavior is this:

setInterval(function () {
    var test = [];
    for (var i = 0; i < 100000; i++) {
        test.push(i);
    }
    var data = {
        test: test
    };
   // console.log(data);
}, 100);

When i comment out the console.log no issues happen but adding it would cause the memory to keep increasing until eventually the app crashes due to an out of memory exception.

Regards, Omar.

[Almost-Done]

I've fixed both issues in the develop/websockets branch.

The first issue was caused by failing to delete the data allocated for an external ArrayBuffer and it was affecting XHRs as well. I fixed the leak and also switched XHR and WebSockets to use simple ArrayBuffers

The second issue was caused by console keeping a log of all messages in memory. I removed the code to store the log history.

[Almost-Done]

PS: Thank you for investigating and reporting this issue. Please let me know if you see memory leaks with the fixes above.

[omaralcheikh]

Hello Cristi,

There are no more memory leaks in both cases. However with sending a buffer i am getting an error "Failure in file \holojshost\websocket.cpp, line 252". This happens if i increase the size of the buffer or if i send multiple buffers.

Also another memory issue i came across is in your chat sample. On every chat received you are removing the text geometry and then re-adding it with new text. It seems when removing the mesh it is not being removed from memory which is causing the same issue as the previous ones mentioned. I did not try it again on your recent fix.

[swomack]

Hello Cristi,

After this commit, i am getting error in XMLHttpRequest if the responseType is arraybuffer. Here is the log -

Failure in file d:\works\holojs\holojs\holojshost\xmlhttprequest.cpp, line 159

[Almost-Done]

I hopefully fixed the issue with receive failing for ArrayBuffers. The new function I added CreateArrayBufferFromBuffer was missing a "return true" at the bottom and the caller got random return values, sometimes "false", causing the errors above.

I fixed the geometry leak in the chat sample; when replacing the text, the old text geometry must be disposed with TextGeometry.dispose()

[omaralcheikh]

Thank you Cristi, its working great now and memory is stable. Please note that i get this error sometimes:

Exception thrown at 0x77442552 in ThreeJSApp-Vs2017.exe: Microsoft C++ exception: Js::RejitException at memory location 0x087FF117.

but it seems that it doesn't affect anything and the script just keeps running normally.