immersivecognition / unity-experiment-framework

UXF - Framework for creating human behaviour experiments in Unity
https://immersivecognition.github.io/unity-experiment-framework/
MIT License
215 stars 41 forks source link

DynamoDB possible memory issue #65

Closed cofficer closed 3 years ago

cofficer commented 3 years ago

I'm running an experiment with DynamoDB and WebGL which takes about an hour to complete where each trial takes at most ~1min. Halfway through, some people report that the game starts to slow down considerably until unplayable. Is tracking data being stored in memory until the end of the experiment? If so, is there a way I can upload this during breaks and clear the memory to improve performance?

jackbrookes commented 3 years ago

Hi, I have not seen this issue before.

The way it works is the trial data are added to a queue, and they are continuously being uploaded in the background. The items are released from memory as they are uploaded. They should still be being uploaded during the breaks. Anyhow, even an hours worth of data is probably not enough to cause memory issues I suspect.

You could test if this is an issue with UXF file uploading by creating a build that has the AWS DynamoDB uploader Data Handler disabled, and testing if you still get the issue - which will immediately discard items when trial ends.

You can also I believe profile a build even in WebGL https://docs.unity3d.com/Manual/profiler-profiling-applications.html

I would suggest making a test condition that automatically ends your trial after a few seconds so you can easily leave it running and come back an hour later to see where the issue is.

I'll also do a bit of testing myself to see if I can replicate this

cofficer commented 3 years ago

I've tried a few different ways to check what the cause of fps drops may be, both regular profiling and memory profiling. Based on what you said, I think the issue may be based on certain browsers and how they interact with WebGL. Perhaps something to do with garbage collection over time? I have not been able to replicate the issue on several laptops/desktops.

I will require people to use Chrome from now on, hopeing that alleviates some of these reports. I will let you know if I manage to figure out more details.

jackbrookes commented 3 years ago

I did test this and found no slowdown or memory leaks. A test experiment running for 90 mins with 30s trials, recording the position of a moving cube. Ended up with only ~11MB of RAM used

image

cofficer commented 3 years ago

Thank you for testing as well!

jackbrookes commented 3 years ago

I am using Firefox here and it works fine. You could add in some collection of the FPS on each trial.

By default UXF collects browser info, and it will be stored in the ParticipantDetails table. Look for the user_agent string. Maybe you can find the problematic cases.

cofficer commented 3 years ago

Yes good idea, I will add FPS collection. I just checked and the table for ParticipantDetails is created but empty in my DynamoDB. Do you why this might be? Does it need to be enabled somewhere?

cofficer commented 3 years ago

Sorry just saw that it is check box!