search

darknight1050 / SongLoader

GNU General Public License v3.0
28 stars 12 forks source link

Out of Memory Crash because of assets (cover images and audio files) not unloading #15

Open WGzeyu opened 3 years ago

WGzeyu commented 3 years ago

I'm not sure if this is a SongLoader issue, but it has been present in all versions since 1.15.0 (nightly), to 1.16.4 (stable), to 1.17.1 (stable), after switching to SongLoader.

I imported 1000+ songs and even with only the core mod installed, it still crashes, almost always in the custom levels screen, the crash occurs when selecting songs / playlist, and if the operation is fast, it crashes in less than a minute. It does not crash when select/play only OST songs. It's not a specific song problem either, if I reopen the game and select the song directly from the last time it crashed, it doesn't crash again immediately. The crash always happens when switching songs back and forth.

Could it be that there is a problem with one of the core mods? I've made a Chinese guide and answer questions in the Chinese community regularly, and every day there are many people who experience the same crashes as I do, with high frequency.

The characteristics I can think of that many users in the Chinese community have are:

  1. the system language is Simplified Chinese.
  2. the presence of a firewall on the network (I make sure the core Mod is properly installed) (without VPN, it would be impossible to connect to Oculus, and connecting to GitHub is polluted by DNS, so many people use VPN).
  3. there are a lot of songs (about 1000+) and many people copy directly to SongLoader directory to import songs.
  4. there are many songs in the info.dat that contain Chinese or Japanese.
WGzeyu commented 3 years ago

Is there anywhere I can find the crash log?

WGzeyu commented 3 years ago

I posted a video. https://youtu.be/HoyjODHM8n0 To quickly reproduce the problem so click faster, in the usual even if not operating so fast will often crash, just does not appear so fast.

darknight1050 commented 3 years ago

This is a known bug, unfortunately I didn't find a way to fix it yet. To explain it simple the game doesn't unload the cover images of songs, so the RAM will get filled up with that. I think it is actually a base game bug as it even happens on PC.

darknight1050 commented 3 years ago

Reopen so people don't report twice.

WGzeyu commented 3 years ago

I haven't encountered this on PC, maybe because of the larger RAM (32GB)? I'll try to make a script to compress the cover image and see if it will reduce the crash frequency.

WGzeyu commented 3 years ago

The cover doesn't seem to affect the song hash and I should be free to change it?

darknight1050 commented 3 years ago

Yeah on PC it won't crash but in Task Manager you clearly see RAM usage go up. I got to 8GB once when testimg this iirc.

As for testing even if the hash changes the song will still load but it will load it as a new song ofc.

WGzeyu commented 3 years ago

I just found out that the cover image file is not very big, and my songs add up to about 400MB. Should I focus on resolution, not file size, when compressing images?

darknight1050 commented 3 years ago

I don't know actually you have to try.

WGzeyu commented 3 years ago

I don't think it's the cover image filling up RAM that causes the crash. My total song cover image is 467MB, just now I tried to compress the image, first to 384x384, jpg (5%), total 9.3MB for all images, still crashes, then I tried 128x128, jpg (95%), total 21.2MB for all images, still crashes. I'll try 32x32 again later

darknight1050 commented 3 years ago

Mhh but what else could it be? Btw you can use https://developer.oculus.com/downloads/package/ovr-metrics-tool/ on you quest to see RAM usage.

WGzeyu commented 3 years ago

Mhh but what else could it be? Btw you can use https://developer.oculus.com/downloads/package/ovr-metrics-tool/ on you quest to see RAM usage.

Just tested using this and it is indeed a RAM usage problem, but not just the cover image. Tested the original cover file (467MB total) and compressed to 32px (4.2MB total) separately. First, I scrolled the song list to the bottom without selecting the song, just loading the cover, and after scrolling to the bottom the original image took up most of the RAM, while the compressed image only took up about half. After that I clicked on the selected songs one by one and switched back and forth between songs, at which point the available memory continued to drop and both sides soon crashed. The difference is probably that after compressing the image I can click on more songs before crashing.

I tried to record the video, but unfortunately the OVR Metrics Tool wouldn't record in. original image: https://www.youtube.com/watch?v=HORYgpmUcCk compressed image: https://www.youtube.com/watch?v=hNQZSoAtbkA

darknight1050 commented 3 years ago

Yeah that makes sense, when you click the song it loads the audio file so that needs RAM aswell of course.

Unfortunately this is difficult to fix. I'll have to see what I can do.

RedBrumbler commented 3 years ago

Just because on your filesystem the images are only 30 mb or so, doesnt have to mean that in game they are that too

RedBrumbler commented 3 years ago

Also I seem to recall this is a base game issue too on pc, but usually people don't have that many songs that it causes issues there

0x4f5da2 commented 2 years ago

My game also crash because of OOM when switching between different songs. However, in a small number of cases, the game crashes with sufficient memory (> 1000MB). Don't know why, the log(around line 13684) shows it may related to a segment fault.

10-04 01:34:46.218 11309 11337 I QuestHook[PinkCore|v1.6.1]: Looking for characteristic: Expert
10-04 01:34:46.218 11309 11337 I QuestHook[PinkCore|v1.6.1]: Looking for diff: Expert
10-04 01:34:46.218 11309 11337 I QuestHook[PinkCore|v1.6.1]: Found CharacteristicName: S
10-04 01:34:46.218 11309 11337 I QuestHook[PinkCore|v1.6.1]: Found diffstring: E
10-04 01:34:46.218 11309 11337 I QuestHook[PinkCore|v1.6.1]: Looking for characteristic: Expert
10-04 01:34:46.218 11309 11337 I QuestHook[PinkCore|v1.6.1]: Looking for diff: Expert
10-04 01:34:46.218 11309 11337 I QuestHook[PinkCore|v1.6.1]: Found CharacteristicName: S
10-04 01:34:46.218 11309 11337 I QuestHook[PinkCore|v1.6.1]: Found diffstring: E
10-04 01:34:46.218 11309 11337 I QuestHook[PinkCore|v1.6.1]: There was custom data!
10-04 01:34:46.218 11309 11337 I QuestHook[PinkCore|v1.6.1]: Extracting Requirements
10-04 01:34:46.218 11309 11337 I QuestHook[PinkCore|v1.6.1]: Extracting Suggestions
10-04 01:34:46.218 11309 11337 I QuestHook[PinkCore|v1.6.1]: Handler ptr: 0x6f1fcf6ea0
10-04 01:34:46.219 11309 11337 I QuestHook[PinkCore|v1.6.1]: (RequirementElement) Requirement Mapping Extensions is installed: 0, required: 0, suggested: 0
10-04 01:34:46.237  1660  1805 I GuardianInput: RightTrigger up
10-04 01:34:46.281 11309 11337 E CRASH   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
10-04 01:34:46.281 11309 11337 E CRASH   : Version '2019.4.28f1 (1381962e9d08)', Build type 'Release', Scripting Backend 'il2cpp', CPU 'arm64-v8a'
10-04 01:34:46.281 11309 11337 E CRASH   : Build fingerprint: 'oculus/hollywood/hollywood:10/QQ3A.200805.001/19130100199900000:user/release-keys'
10-04 01:34:46.281 11309 11337 E CRASH   : Revision: '0'
10-04 01:34:46.281 11309 11337 E CRASH   : ABI: 'arm64'
10-04 01:34:46.281 11309 11337 E CRASH   : Timestamp: 2021-10-04 01:34:46+0800
10-04 01:34:46.281 11309 11337 E CRASH   : pid: 11309, tid: 11337, name: UnityMain  >>> com.beatgames.beatsaber <<<
10-04 01:34:46.281 11309 11337 E CRASH   : uid: 10077
10-04 01:34:46.281 11309 11337 E CRASH   : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
10-04 01:34:46.281 11309 11337 E CRASH   : Cause: null pointer dereference
10-04 01:34:46.281 11309 11337 E CRASH   :     x0  0000000000000000  x1  0000000000000000  x2  0000000000000010  x3  0000000000000002
10-04 01:34:46.281 11309 11337 E CRASH   :     x4  0000006eebbf2678  x5  0000000000000004  x6  0000000000000000  x7  7f7f7f7f7f7fff7f
10-04 01:34:46.281 11309 11337 E CRASH   :     x8  00000000ffffffff  x9  0000000000000000  x10 0000000000000001  x11 0000000000000000
10-04 01:34:46.281 11309 11337 E CRASH   :     x12 0000006eebbf2000  x13 0000000000000020  x14 fffffffffc000000  x15 0008000000000000
10-04 01:34:46.281 11309 11337 E CRASH   :     x16 00000070be3a1bf8  x17 00000071a9200180  x18 00000070ae763000  x19 0000006eebc66ca0
10-04 01:34:46.281 11309 11337 E CRASH   :     x20 0000000000000001  x21 0000000000000001  x22 0000006eebc66ca0  x23 000000000000006b
10-04 01:34:46.281 11309 11337 E CRASH   :     x24 00000070be1addc8  x25 0000000000000001  x26 000000000000000c  x27 00000070b5449ed9
10-04 01:34:46.281 11309 11337 E CRASH   :     x28 00000070b5449ef0  x29 00000070c0c3c690
10-04 01:34:46.281 11309 11337 E CRASH   :     sp  00000070c0c3c690  lr  00000070bdfa19dc  pc  0000000000000000
10-04 01:34:46.329  1721 10865 I TimeWarp: SetSchedFifo( tid=11410, pol=2, pri=0 ) succeeded
10-04 01:34:46.394  1660  1805 I GuardianInput: RightTrigger down
BurgerBurgerBurgerBurger commented 2 years ago

I got this same problem a year later and it seems that it is the clock mod causing this (at least for me) as I have deleted all my mods slowly added them back one by one and as soon as I delete clock mod it works again. I reinstalled it again to make sure and custom songs stopped working again. I'm not sure this is the same reason your custom songs are not working.

szl0834 commented 2 years ago

I have the same problem, but I only have 300 song loaded. And I tried to use OVR Metrics Tool to monitoring the RAM usage, but it seems that Meta's update make it available to use.