Very low FPS (< 10) in Song selection

[cRaZy-bisCuiT]

Actual behaviour

When trying to select a song, a lot of them cause the FPS to drop below 10. It's very odd, since when I start the song the FPS go back to 60.

Expected behaviour

Selecting a song shouldn't make FPS drop that much.

Steps to reproduce

1. Open the Game
2. Select Players
3. Skimming through the song selection

Details

Provide some additional information:

• USDX version: latest git as of 2017_11_09
• Operating System + version: 4.13.11-1-ARCH
• Graphics:

▶ lspci -nnk | grep -i VGA -A2 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/580] [1002:67df] (rev c7) Subsystem: Micro-Star International Co., Ltd. [MSI] Radeon RX 480 [1462:3413] Kernel driver in use: amdgpu ~
▶ glxinfo | grep 'OpenGL version string' OpenGL version string: 3.0 Mesa 17.2.4

The same happens on Raspberry Pi as well as Core2Duo + Intel GMA 4500MHD in Ubuntu 17.04.4

[cRaZy-bisCuiT]

Actually I now know the cause of the problem: When a certain number of songs in the library is exceeded, this problems occurs.

If you press 'J' to apply a filter, e.g. searching for an artist or keyword, the stutter is gone.

Can we make the performance of the song selection better somehow? The longer Ultrastar exists, the bigger people's libraries will be and and therefore the problem will occur to more and more people.

[cRaZy-bisCuiT]

The issue still persists. It does not make a difference if I use a system with Intel 3570K@4,4Ghz and 16 GB RAM or a RPI4: Bad performance is hit in the song selection whenever the library seems to big.

One new observation: Only 700 Mb / 4 GB RAM are used when running the game. Is the RAM usage limited somewhere? I feel like the game could not really cache the whole database due to RAM usage limitations.

[basisbit]

Is your song content stored on a HDD or SSD? When Skimming through the song selection, the image files for these songs have to be read from the storage device, parsed and then transformed to an uncompressed image. These images are not cached anywhere because otherwise, the memory requirement would grow depending on song library size. File system caching is still used by the operating system if there is spare unused RAM left.

The performance is still more than good enough for all the systems which I saw so far. Improving this would require quite a bit of code change. I personally rather spend my free time continuing UltraStar Play development (which by the way already includes very fast song loading in background - it allows to browse + search through the songs even while it is still loading more song content). Of course anyone may feel free to improve this in UltraStar Deluxe.

[s09bQ5]

I see... You removed the caching of uncompressed cover thumbnails in the sqlite database in 2016.

[basisbit]

yes, because that was very buggy and did not scale well for big song databases.

[cRaZy-bisCuiT]

Actually it does not make any difference if I use a SSD, HDD or if the Songs are located on a NAS. Of course I would assume the SSD should be much faster, but this it not the case. The bottleneck seems to be somewhere else. Is there a memory limit the app could use either by architecture contraints or even something that is set somewhere? I would want either the OS or the game to rather use my RAM to improve performance instead of leaving it free...

I tested this with the same library (7000+ songs) with somethimes videos, sometimes not. All the time I limit the given results (e.g. search for an Artist like ABBA -> only e.g. 80 songs are filtered) browsing will become very smooth again. Btw, by slow I mean it starts to stutter (e.g. current video preview stutters) and it's hard to navigate. If I press R then, I'd had to wait a few seconds for a new song to show up. If I search by pressing J, there's a delay of a few seconds until the text is shown/filter applied. Like shown in the first post: Dropping from a stable 60 FPS (probably VSYNC) to under 10 FPS most of the time.

If there's no viable way to improve this performance I could see it does not make sense to put a lot of effort into this game anymore, since USPlay might be the go-to game any time soon.

I could provide some logs / performance metrics etc. if you tell me what you need. If there's nothing to do here, feel free to either leave it open for someone to take care of or close it with WONT FIX.

[basisbit]

that actually sounds like a bug. could you please provide the error.log?

[s09bQ5]

According to OProfile it's TCatSongs.VisibleIndex that kills the performance:

Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (core_p Core cycles when core is not halted) count 100000
samples  %        image name               app name                 symbol name
1428511  44.5100  vmlinux                  vmlinux                  intel_idle
800926   24.9555  ultrastardx              ultrastardx              USONGS$_$TCATSONGS_$__$$_VISIBLEINDEX$LONGINT$$LONGINT
376580   11.7336  libavcodec.so.56.60.100  ultrastardx              /usr/lib64/libavcodec.so.56.60.100
68593     2.1372  libavutil.so.54.31.100   ultrastardx              /usr/lib64/libavutil.so.54.31.100
42660     1.3292  libnvidia-glcore.so.340.93 ultrastardx              /usr/lib64/libnvidia-glcore.so.340.93
40462     1.2607  libavformat.so.56.40.101 ultrastardx              /usr/lib64/libavformat.so.56.40.101
32895     1.0250  libc-2.18.so             ultrastardx              __GI___strcmp_ssse3
18619     0.5801  libm-2.18.so             ultrastardx              __sin_sse2
13551     0.4222  libc-2.18.so             ultrastardx              _int_malloc
12652     0.3942  libc-2.18.so             ultrastardx              __memcpy_ssse3
10608     0.3305  ultrastardx              ultrastardx              USCREENSONG$_$TSCREENSONG_$__$$_SETROULETTESCROLL
10552     0.3288  ultrastardx              ultrastardx              USCREENSONG$_$TSCREENSONG_$__$$_SETROULETTESCROLLREFRESH

It should be easy to optimize this by saving the position of a song in the list of visible songs inside TSong.

[s09bQ5]

Please test if this is fixed by #534