Game occasionally freezing/lagging

[opl-]

I have received 3 reports of game occasionally hanging while using the plugin. Here's a summary of those reports:

• Amanatsu says their game keeps freezing up every few minutes. They provided this clip as an example. Reselim and I have analyzed a part of the associated VOD and found lag happening at these timestamps (this list is not conclusive): 0:40:06-0:40:09, 2:33:36-2:33:40, 2:39:26-2:39:29, 2:43:14-2:43:17 and 2:46:41-2:46:45. During all of those the music kept playing, but no user inputs were being processed. At 2:33:40 we can see the pause menu being triggered after the game unfroze. The game also doesn't trigger the fail pitch change until it unfreezes. This suggests that the Update methods aren't being called while the game is frozen. Additionally it's worth noting that there doesn't appear to be a clear pattern for when the lag occurs (if you can spot one, please do comment). Amanatsu says that the freezes started after they installed the plugin.

• souk was mentioned by Amanatsu in a Discord message in which they say "souk and I are both experiencing HMD freezing while using the plugin/game freezes for several seconds at a time, then resutms but you fail immediately as you've missed a bunch of notes". According to Amanatsu, souk started experiencing the freezing after installing the plugin. I have not established contact with souk so far.

• Reselim says they experienced the game freezing up during one session. They described it as "minor lag issues for a single session".

My only suspected reason for such freezes was the websocket library the plugin uses, however after analyzing it, it appears to be using asynchronous methods when managing connections.

Due to lack of any solid evidence that the lag was caused by the plugin (can't reproduce the issue reliably, only appears to affect 2 users) and lack of any leads as to what may be causing the freezing, I will not be attempting to fix this issue for now. However, if you are experiencing or know someone who is experiencing it, I would like you to comment on this issue or ask them to do so.

Please include this information in your comment:

• does the issue happen: every session, only some sessions, only once...
• how often does the issue happen: every few minutes, once an hour...
• what plugins you have installed
• if the issue happens without the HTTP Status plugin
• if you were streaming/recording when the issue happened
• if the issue happens when you're not streaming/recording

[opl-]

Received an update from Amanatsu. They reinstalled their OS and tried streaming again. They reported the issue being fixed for them.

[Mattches77]

I know this issue is about a year old, but I'd like to report my experience with it. I'm getting very brief, intermittent freezing while http status is active in my OBS. Just enough of a freeze to miss a note, and at random time intervals (every 30sec to 1min?). It's easily reproduced/confirmed by hiding the http status source in my obs scene - freezes while it's used in the scene, none as soon as I hide / turn off the source. Whether or not I'm recording or streaming doesn't effect it, just having OBS open and it in the scene is enough. While I haven't gone as far as reinstalling OS, reinstalling SteamVR and Beat Saber had no effect. Writing because I love the plugin and want to be able to use it while recording but have to hold off for now.

[ECrusher]

My experience is the same as Mattaches. I discovered it was HTTP Status by removing all my mods then readding them one by one. It seems there is a pattern to the lag spikes, but it's on a per song basis. Meaning that for each song, it will happen on its own, semi-consistent timing. The songs I was using for testing were $100 Bills and Big Metal Shoe by Joetastic. $100 Bills would do it around every 30 seconds. Big Metal Shoe does it around every 20 seconds.

You do not need to be streaming for this to happen; just have OBS open. I also tried multiple versions of OBS to be safe (OBS Studio 64-bit 23.2.1 and Streamlabs OBS 0.16.3).

Also hoping that this issue can get resolved because I think this is one of the best Beat Saber mods for streaming and practicing.

Thank you for your hard work!

[BaliBalo]

Potential idea for performance improvements: maybe switch to a "subscription" system where each overlay would indicate what events they want to receive. This would avoid for example sending all the lighting events if they are not needed by the overlay. That could cause retro-compatibility issues but if you assume overlays are subscribed to everything until they specify a list of events, then it should still work the same.

[opl-]

That idea has been on the table for quite a while now. Current ideal implementation would simply send all the events that were previously sent by default unless a query parameter is provided while connecting.

That being said the whole event system is admittedly written pretty poorly. monkeymanboy recently pointed out to me that the mod is creating new JSON objects for every event instead of reusing them, which I forgot about. Initially the GameStatus object was supposed to store all of that data in order to both make it possible to take a snapshot of the state in case some event required it and to prevent constantly creating new JSON objects, but at some point that approach ended up not working out and I switched to the current, pretty awful system.

I want to rewrite this at some point, but haven't really had much motivation recently to do so and I don't want to mess it up again.

[johndoe31415]

Hi there, just browsing the bugtracker because I was looking if a 1.4.0 compatible version was already realeased -- this issue caught my eye.

I'm using beatsaber-http-status to record/historize data. No streaming at all and a dedicated machine that does nothing else but record the websockets stream (and write it to disk after the song's complete, i.e., up until that point everything is in-memory and blazing fast).

Occasionally I have described what you're describing here as well. I've noticed a pattern for me where the websockets stream was repeatedly disconnected for no apparent reason. The lag in the game occurred when the connection reconnected and the initial status message was sent to the client, apparently because in some of the custom songs I'm playing the image thumbnail (transmitted as base64-encoded blob inside the status message) is ridiculously large (easily 600kB, 700kB for the "songCover" item alone). I'm fairly sure this explains the lag upon reconnect. Doesn't explain why the connection is lost in the first place, unfortunately.

EDIT: Huge fan of your plugin, thanks a bunch for all the great work. It's fantastic!

[AhiVT]

Lag every X seconds sounds like garbage collection. Reuse of objects and a smaller memory footprint may help combat this issue.

[gliderman]

The other day I was running Beat Saber 1.7.0 and ran into this problem (narrowed down to just this mod, and only when OBS was open with a connection to the mod). The other comment mentioning garbage collection made me install the TrashMan mod which allows the garbage collector to be tweaked in the game. I set it so it doesn't garbage collect while in a song, and all the issues with lag spikes every 20-30 seconds disappeared. I have 32 GB of RAM in my system, so I wasn't worried about running out of memory.

[Mijago]

The websocket sends the events synchronously, so it's blocking the whole process until the packet is sent. I fixed it easily by replacing 'Send' with 'SendAsync' with an Empty callback (as I don't need it).

In general, all http/websocket procedures should happen in a different threat. In this case, editing the sending commands of the websocket should be enough, as I don't use the http calls (yet) I can't say if it requires modification too, but I bet it does.

[beaudamion]

I know this was supposedly fixed in October but the recent release of Beat Saber 1.13.4 the stutter / Lag happens for me and another user. I'm not sure if it's an issue with the new Beat Saber release and websockets 1.0.4 as I know datapuller uses the same thing and I get the exact same issue as well. If I disable the overlay the stutter/lag goes away. Seems to be every 15 or 30 seconds.

[opl-]

Yesterday while testing I had some issues with audio lagging (sometimes completely cutting out for about a second) while the game kept running fine. Two other modders reported having similar issues, so for now I'm assuming this is an issue with Beat Saber and not this mod. If I have some energy to play today, I'll try to see if this is an issue in the vanilla game too.

[opl-]

I just did some testing on my machine, after a reboot and without a browser running. The vanilla game seemed to be running fine (though I only ran it for a bit right at the beginning). Modded seemed stuttery, both with and without HTTP Status installed.

Since I'm on Linux and have basically nothing in terms of usable performance testing tools, I used the sophisticated method of counting dips on the SteamVR performance graph. Both with and without the mod performance seemed to be about equally as bad, with the game running mostly fine until it dipped into orange for a number of frames, sometimes throwing in an occasional magenta frame to spice it up.

In the end, I don't have the tools to test this properly and it doesn't seem to be isolated to when I'm running HTTP Status, so there's not much I can do here without losing my sanity, or switching to Windows and wasting an eternity waiting for all the downloads I'd need to do.

Mods I was running: BeatSaverDownloader, BS Utils, BSML, ScoreSaber, SongBrowser, SongCore, SongDataCore, and of course (sometimes) HTTP Status.

[ErisApps]

Okay, just gonna respond to this as well to give some additional info. Since Beat Saber 1.13.4+, the game engine Unity has been updated to 2019.4.18f1 and due to that change, the GC mode has been changed (no clue whether it was on purpose or not). Anyways... it's possible to return to the original GC mode by either using Auros's and Umby's GC Fixer or by manually adding gc-max-time-slice=30 to Beat Saber_Data/boot.config (which is essentially what the former solution does)

[Macil]

I was having some stuttering in Beat Saber today, and I thought it was especially happening each time I cut blocks while I had a websocket connection running, which made me really suspicious that the mod might be doing blocking IO on the main thread each time it sent out websocket events. (This could randomly take longer especially when the websocket client is being slow to consume the connection.) Looking at the code, I see that WebSocketBehavior.Send gets called while handling game events, which appears to be a synchronous/blocking IO call. Using WebSocketBehavior.SendAsync (with a null completed parameter - edit: actually this parameter would be important to use) instead would clear that problem up. This might be the whole problem.

[opl-]

Yeah, that's #54 that I definitely didn't end up forgetting about. I was worried about messages arriving out of order when blindly using SendAsync.From the little research I did into C# threading it seems that'd be a possible race condition with the default behavior, so I wanted to write a bit more code that'd ensure delivery order.

Feel free to compile the mod yourself if you can to see if it helps in your case and report back. I also want to add that I'm unable to launch the game when it's modded since the last update and we're still unsure why, which makes modding a little difficult.

[Macil]

Oh good call. I haven't worked on a beat saber mod yet but I could give that a shot.

[opl-]

Macil's PR landed in v1.15.1 and should fix this issue. If anyone has a similar issue with v1.15.1 or later installed, please open a new issue.