search

ppy / osu

rhythm is just a *click* away!
https://osu.ppy.sh
MIT License
15.28k stars 2.27k forks source link

Unexpected frame stutters at regular intervals (GC related) #11800

Closed pinembour closed 1 year ago

pinembour commented 3 years ago

Describe the bug: A few times per map I encounter a stutter that makes frame time for one frame spike. The screenshot shows the issue in mania but I encounter it in every game mode. From what I saw, nothing in the logs seems to happen at the same time as the stutter.

I can reproduce this issue with stock or custom kernel, 5.10 or 5.11, amd gpu or nvidia, built from source or using appimage.

Screenshots or videos showing encountered issue: Screenshot from 2021-02-17 09-38-32

osu!lazer version: 2021.212.0.r106.86faa7f465-1, reproductible with 2021.109.0-1 and anything in between, did not try anything earlier

Logs:

logs.zip

peppy commented 3 years ago

Thanks for the trace. The attached trace still shows a long gen0 collection:

20210302 102924 (Parallels Desktop app)

This might be due to drawables being created somewhere though, it's a very hard one to discern. We'll fix the pieces of the puzzle that we found (follow https://github.com/ppy/osu-framework/issues/4229 and https://github.com/ppy/osu-framework/issues/4228 if interested) and try again after those are resolved.

peppy commented 3 years ago

Are you both able to test again on the latest master? We've made some further improvements.

pinembour commented 3 years ago

trace.zip Here is a trace on latest master. Stayed on the home menu moving my pointer until ~1m50s ( There is a big stutter at this time which comes from loading the next song ), then moved through the menus and selected a song until 2m30s, and the rest is a osu!taiko map in auto play. I didn't notice any stutter in the home menu, but there were still some while in a map.

peppy commented 3 years ago

Thanks. We're still aware of issues during gameplay which will be resolved with some more work.

Micha-ohne-el commented 3 years ago

Here are two traces, one with the performance graph visible, the other with it completely turned off. You didn't specify, so I recorded both. As always, I was just in the main menu, moving my cursor around if anything. I noticed stutters when I could see the graph, but I didn't catch any when it wasn't visible, but there may still be some (hence why that trace is longer, just in case). traces.zip And for clarity: Latest master (a792c3f13c21d207378e8dbdf151e84ad7e033ef) built with dotnet run --project osu.Desktop -c Release.

Micha-ohne-el commented 3 years ago

Wait, that isn't the latest master. Hold on, I gotta figure this out.

peppy commented 3 years ago

Please test with the graph not visible. we are aware of one remaining issue when it is visible. you can leave the fps counters visible, though.

Micha-ohne-el commented 3 years ago

Alright, now the actual latest master (dc7573f7c7165d6f13bba7eae2897df723d83b77). Sorry about that. And this time only with the graph hidden: trace.nettrace.zip

peppy commented 3 years ago

That log is looking pretty clean now.

smoogipoo commented 3 years ago

Hey all, if you're able to compile osu! + o!f locally, I'd like you to try out https://github.com/ppy/osu-framework/pull/4298 and see if it helps. If not, it should be in the next osu! build.

pinembour commented 3 years ago

I tried compiling following what is written on the wiki but it's failing, see here. I guess I will wait for the next build.

LeaPhant commented 3 years ago

Hey all, if you're able to compile osu! + o!f locally, I'd like you to try out ppy/osu-framework#4298 and see if it helps. If not, it should be in the next osu! build.

I assume this is in the newest release (2021.323.0) and it didn't change anything for me sadly. I also tried it separately by following the instructions and building myself and the stutters still occur.

what DE are you using? I'm using KDE Plasma, maybe that could be a factor (though I doubt it)?

I am also on KDE Plasma, but I tried GNOME and had the same issues.

Micha-ohne-el commented 3 years ago

The latest release does not seem to improve performance during gameplay for me, not sure about other parts of the UI. I can't seem to build osu! with a local framework, and I don't have much time to find a solution for this, unfortunately. These are the errors I get upon running dotnet run --project osu.Desktop -c Release: errors.txt I already ran the shell script that configures the osu! solution to use the local build of the framework. That ran with no errors. I'm on Dotnet version 5.0.104.

peppy commented 3 years ago

We've made some pretty large improvements on this front. Things are still not perfect but I'm going to remove this from a milestone task for the time being. We still have some ongoing tasks which will further improve the situation.

Please feel free to report back with further profiling in subsequent builds.

smoogipoo commented 3 years ago

Just posting back for future reference...

https://osu.ppy.sh/beatmapsets/519505#osu/1906276 played with osu!mania is a good way to see the stutters in full effect: image

Morodar commented 3 years ago

I experience some lag spikes while playing beatmaps. I haven't noticed those lag spikes for quite a while.

It also appears to happen more often when I limit the game to 160FPS via NVIDIA Control Panel.

Screenshots or videos showing encountered issue:

YouTube Lags occur at

osu!lazer version: 2021.707.0-lazer

Logs: Beatmap starts at 15:39:53 and ends at 15:40:31 performance-update.log runtime.log updater.log database.log network.log performance.log performance-draw.log performance-input.log performance-audio.log

smoogipoo commented 3 years ago

Hard to say - nothing's changed in the last little while that would make them more prominent, but these spikes have been occurring for a long time now. It would be better to show the frame time by pressing Ctrl+F11 once more to get the graphs to show.

You could also try running osu! with COMPlus_GCGen0MaxBudget=600000 set as an environment variable.

Morodar commented 3 years ago

Summary

Just the next days I was not able to reproduce it. It's so weird.

Probably some driver issues?
I'll try to report it whenever it occurs again (providing logs and Ctrl+F11).

For the sake of completeness, here are two recordings using the same pc, same osu version etc..
Those recordings do not show the lag spikes.
I'm pretty sure they don't help you. :/

At least I can tell you that osu! is running at 600-700FPS on my machine (in multi-thread mode and when I'm not recording) 😄

Single threaded video

YouTube performance-input.log performance-update.log runtime.log network.log performance.log performance-audio.log performance-draw.log

Multi threaded video

YouTube network.log performance.log performance-audio.log performance-draw.log performance-input.log performance-update.log runtime.log

smoogipoo commented 3 years ago

You don't need to keep reporting back. We know with absolutely 100% confidence that there is an issue, even if you're not experiencing it.

hez2010 commented 3 years ago

Does this help? This PR fix an issue which is causing periodical long GC on gen 1, to verify whether it helps we need either use nightly build or 6.0 RTM for testing.

smoogipoo commented 3 years ago

Improvement looks to be marginal at best. Left is net6-rc1, right is compiled from master: image

As I mentioned in the thread, there seems to be a difference in how the GC tunes its gen1 heap size on Linux vs Windows, resulting in making it too large on Linux and triggering collections less frequently than we'd like. That's the primary issue here.

The .NET team have yet to respond, unfortunately...

hez2010 commented 3 years ago

How about the new standalone region based GC? (it can be built from source and consumed via setting DOTNET_GCName to the path of built clrgc.dll). It's not finished yet but worth a try in my opinion.

smoogipoo commented 3 years ago

Regions currently segfaults during gameplay. I also posted a repro for that in the comment I linked.

smoogipoo commented 2 years ago

An update on regions - they've since fixed the segfaults but it doesn't look like regions are helping, unfortunately. I still have to do my own investigation because it seems like Linux vs Windows allocates different budgets for Gen0, which is probably a bug that's somehow translating to regions as well.

DanielPower commented 2 years ago

As a workaround until this is resolved, is there some way to prevent garbage collection during gameplay (provided the user has enough memory)? It would be preferable to have a massive garbage collection spike when the song is over, rather than periodic spikes throughout the song. I'm experiencing this issue on both Linux and Windows, which makes playing on Lazer infeasible currently.

Also I noticed that on June 13th, Peppy added the tag "platform:linux" to this issue. Though multiple users have reported this occurring on Windows. Including #13969 which was closed as a duplicate of this. So it seems this is not platform-specific.

peppy commented 2 years ago

If you read the issue you linked, you will notice that I say it is in no way related to this. It is also related to vRAM being exceeded as the user points out.

There are no required workarounds for this issue on windows because it doesn't occur. If you are having performance issues, please open a new discussion thread. Make sure to test using the default skin and settings before doing so, as we have found some skins can cause issues due to excessively large resources.

DanielPower commented 1 year ago

I'm not sure exactly which release resolved the issue. But I'm happy to report that at some point between 2022.1101 and 2022.1117, all stuttering issues have been resolved for me on Linux.

peppy commented 1 year ago

Great to hear! Most likely the latest release.

peppy commented 1 year ago

Going to tentatively close this for the time being.

NimaQu commented 1 year ago

Seems i have the same problem on Windows, screenshot: image logs: performance.log

version: 2022.1228.0

peppy commented 1 year ago

That's not GC. Swap buffer can be caused by anything. Definitely not this issue. Please wait for future performance updates.