large malloc when launching Portal 2

[fberg]

gdb output:

$ GAME_DEBUGGER=gdb ./portal2.sh -game portal2 -steam
GNU gdb (Gentoo 7.7 p1) 7.7
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /mnt/raid/Games/steam-library/SteamApps/common/Portal 2/portal2_linux...(no debugging symbols found)...done.
warning: Could not load shared library symbols for linux-gate.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0xf7519b40 (LWP 8226)]
[New Thread 0xf7418b40 (LWP 8227)]
[New Thread 0xf7317b40 (LWP 8228)]
[New Thread 0xf7216b40 (LWP 8229)]
[New Thread 0xf7115b40 (LWP 8230)]
[New Thread 0xf7014b40 (LWP 8231)]
[New Thread 0xf6f13b40 (LWP 8232)]
[New Thread 0xf6e12b40 (LWP 8233)]
SDL video target is 'x11'
SDL video target is 'x11'

SDL failed to create GL compatibility profile (whichProfile=0!
SDL failed to create GL compatibility profile (whichProfile=0!

This system supports the OpenGL extension GL_EXT_framebuffer_object.
This system supports the OpenGL extension GL_EXT_framebuffer_blit.
This system supports the OpenGL extension GL_EXT_framebuffer_multisample.
This system DOES NOT support the OpenGL extension GL_APPLE_fence.
This system supports the OpenGL extension GL_NV_fence.
This system supports the OpenGL extension GL_ARB_sync.
This system supports the OpenGL extension GL_EXT_draw_buffers2.
This system supports the OpenGL extension GL_EXT_bindable_uniform.
This system DOES NOT support the OpenGL extension GL_APPLE_flush_buffer_range.
This system supports the OpenGL extension GL_ARB_map_buffer_range.
This system supports the OpenGL extension GL_ARB_vertex_buffer_object.
This system supports the OpenGL extension GL_ARB_occlusion_query.
This system DOES NOT support the OpenGL extension GL_APPLE_texture_range.
This system DOES NOT support the OpenGL extension GL_APPLE_client_storage.
This system DOES NOT support the OpenGL extension GL_ARB_uniform_buffer.
This system supports the OpenGL extension GL_ARB_vertex_array_bgra.
This system supports the OpenGL extension GL_EXT_vertex_array_bgra.
This system supports the OpenGL extension GL_ARB_framebuffer_object.
This system DOES NOT support the OpenGL extension GL_GREMEDY_string_marker.
This system supports the OpenGL extension GL_ARB_debug_output.
This system supports the OpenGL extension GL_EXT_direct_state_access.
This system supports the OpenGL extension GL_NV_bindless_texture.
This system DOES NOT support the OpenGL extension GL_AMD_pinned_memory.
This system supports the OpenGL extension GL_EXT_framebuffer_multisample_blit_scaled.
This system supports the OpenGL extension GL_EXT_texture_sRGB_decode.
This system supports the OpenGL extension GL_NVX_gpu_memory_info.
This system DOES NOT support the OpenGL extension GL_ATI_meminfo.
This system supports the OpenGL extension GL_EXT_texture_compression_s3tc.
This system supports the OpenGL extension GL_EXT_texture_compression_dxt1.
This system DOES NOT support the OpenGL extension GL_ANGLE_texture_compression_dxt3.
This system DOES NOT support the OpenGL extension GL_ANGLE_texture_compression_dxt5.
This system supports the OpenGL extension GLX_EXT_swap_control_tear.
Using breakpad crash handler
Setting breakpad minidump AppID = 620
Forcing breakpad minidump interfaces to load
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
Steam_SetMinidumpSteamID:  Caching Steam ID:  76561198103781699 [API loaded yes]
Steam_SetMinidumpSteamID:  Setting Steam ID:  76561198103781699
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
tcmalloc: large alloc 4294950912 bytes == (nil) @ 

Program received signal SIGTRAP, Trace/breakpoint trap.
0xf7f786dd in ?? () from /mnt/raid/Games/steam-library/SteamApps/common/Portal 2/bin/libtier0.so
Argument list to give program being debugged when it is started is "-game portal2 -steam".

strace says OUT OF MEMORY after the large alloc:

Steam_SetMinidumpSteamID:  Caching Steam ID:  76561198103781699 [API loaded yes]
Steam_SetMinidumpSteamID:  Setting Steam ID:  76561198103781699
tcmalloc: large alloc 4288274432 bytes == (nil) @ 
tcmalloc: large alloc 4288274432 bytes == (nil) @ 
***** OUT OF MEMORY! attempted allocation size: 4288274089 ****

I'm running Gentoo amd64. Does it try to malloc a negative number of bytes? The number 4288274432 leads to 4GiB - 4288274432 = 16384 = 16KiB...

[jimmyhon]

Using breakpad crash handler
Setting breakpad minidump AppID = 620
Forcing breakpad minidump interfaces to load
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
Steam_SetMinidumpSteamID:  Caching Steam ID:  76561197960444411 [API loaded yes]
Steam_SetMinidumpSteamID:  Setting Steam ID:  76561197960444411
Did not detect any valid joysticks.

tcmalloc: large alloc 4289134592 bytes == (nil) @ 
tcmalloc: large alloc 4289134592 bytes == (nil) @ 
Game removed: AppID 620 "Portal 2 (Beta)", ProcID 2719 

Why does it try to make such a large allocation?

Same problem here. Latest ArchLinux amd64 on AMD Phenom II 1090T w/24GB RAM and Nvidia 660 GTX 2GB.

Runs fine on a Macbook Pro 7.1 (with ArchLinux amd64) Core2 8600 w/4GB RAM and Nvidia 320M

[fberg]

The problem went away after I installed Portal 2 on a steam library on another partition. Previously, I had it installed on a RAID, now that I installed it on my system SSD (/home, the same partition where steam is installed), the game works fine.

On a related note, I tried to install TF2 on the RAID today, and gdb again gives large malloc errors. TF2 worked perfectly when I had it installed on /home.

$ LD_LIBRARY_PATH=/mnt/raid/Games/steam-library/SteamApps/common/Team\ Fortress\ 2/bin/ gdb ./hl2_linux 
GNU gdb (Gentoo 7.7 p1) 7.7
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./hl2_linux...(no debugging symbols found)...done.
(gdb) run -game tf
Starting program: /mnt/raid/Games/steam-library/SteamApps/common/Team Fortress 2/hl2_linux -game tf
warning: Could not load shared library symbols for linux-gate.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0xf6efcb40 (LWP 17819)]
[New Thread 0xf66fbb40 (LWP 17820)]
SDL video target is 'x11'
SDL video target is 'x11'
This system supports the OpenGL extension GL_EXT_framebuffer_object.
This system supports the OpenGL extension GL_EXT_framebuffer_blit.
This system supports the OpenGL extension GL_EXT_framebuffer_multisample.
This system DOES NOT support the OpenGL extension GL_APPLE_fence.
This system supports the OpenGL extension GL_NV_fence.
This system supports the OpenGL extension GL_ARB_sync.
This system supports the OpenGL extension GL_EXT_draw_buffers2.
This system supports the OpenGL extension GL_EXT_bindable_uniform.
This system DOES NOT support the OpenGL extension GL_APPLE_flush_buffer_range.
This system supports the OpenGL extension GL_ARB_map_buffer_range.
This system supports the OpenGL extension GL_ARB_vertex_buffer_object.
This system supports the OpenGL extension GL_ARB_occlusion_query.
This system DOES NOT support the OpenGL extension GL_APPLE_texture_range.
This system DOES NOT support the OpenGL extension GL_APPLE_client_storage.
This system DOES NOT support the OpenGL extension GL_ARB_uniform_buffer.
This system supports the OpenGL extension GL_ARB_vertex_array_bgra.
This system supports the OpenGL extension GL_EXT_vertex_array_bgra.
This system supports the OpenGL extension GL_ARB_framebuffer_object.
This system DOES NOT support the OpenGL extension GL_GREMEDY_string_marker.
This system supports the OpenGL extension GL_ARB_debug_output.
This system supports the OpenGL extension GL_EXT_direct_state_access.
This system supports the OpenGL extension GL_NV_bindless_texture.
This system DOES NOT support the OpenGL extension GL_AMD_pinned_memory.
This system supports the OpenGL extension GL_EXT_framebuffer_multisample_blit_scaled.
This system supports the OpenGL extension GL_EXT_texture_sRGB_decode.
This system supports the OpenGL extension GL_NVX_gpu_memory_info.
This system DOES NOT support the OpenGL extension GL_ATI_meminfo.
This system supports the OpenGL extension GL_EXT_texture_compression_s3tc.
This system supports the OpenGL extension GL_EXT_texture_compression_dxt1.
This system DOES NOT support the OpenGL extension GL_ANGLE_texture_compression_dxt3.
This system DOES NOT support the OpenGL extension GL_ANGLE_texture_compression_dxt5.
This system supports the OpenGL extension GLX_EXT_swap_control_tear.
GL_NV_bindless_texture: DISABLED
GL_AMD_pinned_memory: DISABLED
GL_EXT_texture_sRGB_decode: AVAILABLE
Using breakpad crash handler
[S_API FAIL] SteamAPI_Init() failed; no appID found.
Either launch the game from Steam, or put the file steam_appid.txt containing the correct appID in your game folder.
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
Did not detect any valid joysticks.
GL_NVX_gpu_memory_info: AVAILABLE
GL_ATI_meminfo: UNAVAILABLE
GL_NVX_gpu_memory_info: Total Dedicated: 2097152, Total Avail: 2097152, Current Avail: 1456976
GL_MAX_SAMPLES_EXT: 32
[New Thread 0xede8eb40 (LWP 17821)]
CShaderDeviceMgrBase::GetRecommendedConfigurationInfo: CPU speed: 3700 MHz, Processor: GenuineIntel
GlobalMemoryStatus: 4294967295
CShaderDeviceMgrBase::GetRecommendedConfigurationInfo: CPU speed: 3700 MHz, Processor: GenuineIntel
GlobalMemoryStatus: 4294967295
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
tcmalloc: large alloc 4294950912 bytes == (nil) @ 
[New Thread 0xedd8db40 (LWP 17822)]
[New Thread 0xee1fcb40 (LWP 17823)]
[New Thread 0xed386b40 (LWP 17824)]
[New Thread 0xecb85b40 (LWP 17825)]
[0511/202549:ERROR:object_proxy.cc(239)] Failed to call method: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.NetworkManager was not provided by any .service files
[New Thread 0xec384b40 (LWP 17826)]
[New Thread 0xebb83b40 (LWP 17827)]
[0511/202549:WARNING:proxy_service.cc(958)] PAC support disabled because there is no system implementation
[New Thread 0xeb382b40 (LWP 17828)]
IDirect3DDevice9::Create: BackBufWidth: 1920, BackBufHeight: 1200, D3DFMT: 3, BackBufCount: 1, MultisampleType: 0, MultisampleQuality: 0
GL sampler object usage: ENABLED
GL prefer MapBufferRange: NO
[New Thread 0xee1dab40 (LWP 17829)]
[New Thread 0xe65f1b40 (LWP 17830)]
[Thread 0xe65f1b40 (LWP 17830) exited]
[New Thread 0xe65f1b40 (LWP 17831)]

 ##### swap interval = 0     swap limit = 1 #####
Loaded program cache file "glbaseshaders.cfg", total keyvalues: 266, total successfully linked: 266
Could not load program cache file glshaders.cfg
Could not find user GL shader cache file
Precache: Took 474 ms, Vertex 866, Pixel 1369
server.so loaded for "Team Fortress"
[New Thread 0xe2772b40 (LWP 17832)]
[New Thread 0xe2671b40 (LWP 17833)]
[New Thread 0xe2570b40 (LWP 17834)]

Program received signal SIGSEGV, Segmentation fault.
0xf329e15e in ?? () from /mnt/raid/Games/steam-library/SteamApps/common/Team Fortress 2/bin/engine.so

This leads me to believe that this bug is probably related to the Source engine or Steam itself...

[kblaschke]

I've got the same problems on my Gentoo ~amd64 box. tcmalloc is part of the Google Perfomance Tools, and those seem to have some open issues. This one especially looks like a good candidate: https://code.google.com/p/gperftools/issues/detail?id=494

It's possible that Valve linked newer builds of the Source Engine with -ltcmalloc (or a more recent version of the library), causing the problems to show up.

[Gijom]

I've got the same problem on my machine with the following configuration

• Linux Distro: Centos 7
• DM: gnome 3
• Graphic card: NVIDIA Corporation G96GL [Quadro FX 580]
• THE GAME IS ALSO INSTALLED ON A RAID ARRAY (no other partition available to test it on a standard drive.

Can this be really caused by a RAID array ? I fail to understand the link between the support on which the program is installed and the process memory allocation. But still I tought it would be useful to inform you that I have the same problem and a RAID.

[fberg]

Are you by any chance using XFS on the partition? Since I last posted in this thread I have encountered the problem with several other games (i.e. games don't run when installed on the RAID but are fine on my ext4 /home) and I have come to suspect that the problem lies with the inode64 option that is necessary for XFS to support large file systems (> 1 TiB). It seems that some games simply do not support 64bit inodes in their 32bit binaries, an issue which was present in Steam also but was fixed some time ago, see https://github.com/ValveSoftware/steam-for-linux/issues/975.

[Gijom]

Yes indeed I am using xfs, very good point (this is the brand new thing of CentOS 7).

The mount command indicates that the filesystem is mounted with the inode64 option. It seems we now know from where it comes...

So if I understood correctly from the post nothing as been done to solve this issue ? A beta version was mentioned but is it correcting this issue ?

Thanks for your fast help anyway !

[kblaschke]

That's most likely the cause why I've got the same issue, I solely use XFS, and all partitions are larger than 1 TB. I'll try to install an affected game on a smaller USB HDD formatted with ext4, to see if this solves the problem. I found this interesting and quite detailed article which explains the issue and also suggests solutions and workarounds: http://www.tcm.phy.cam.ac.uk/sw/inodes64.html

This would also explain the huge malloc - if the game tries to allocate memory to load a file, and uses the wrong value from the 32 bit stat() call. In addition to _FILE_OFFSET_BITS=64, defining _LARGEFILE64_SOURCE won't hurt.

[fberg]

@Gijom: I currently don't have Portal 2 installed, so I can't verify if this has been fixed, although I doubt it, seeing as this behavior is still present in (the latest "beta" of) TF2. @kblaschke: Thanks for posting the link! I have used the last suggestion (the one preloading inode64.so) to get TF2 to run and I suspect that it would also work with Portal 2.

Of course, this is at best a temporary workaround until Valve provide a fix for this.

[Gijom]

I was also planning on testing this last solution. However I am a bit afraid as some inodes would then be redundant. Do you confirm that this only concern stat and readdir functions (i.e. read only and non destructive functions) ? I would not like to delete a random file of my system just because it as the same 32bit inode than a Portal2 file...

[fberg]

Pardon my ignorance, but shouldn't it be safe as long as the game only uses file names to open/unlink files? The strace log shows that this seems to be the case, and as far as I could find, open() and unlink() only accept file names. Do you have a suggestion as to which system calls I should look for, or another method for detecting this?

But yeah, to be absolutely sure one should probably only try this on file systems which do not contain valuable data.

PS: I installed Portal 2 and the same trick also works...

[Gijom]

hum... I think you are right since anyway the kernel functions only uses file descriptors (created from the file names) and not inodes. I just confounded them for a minute. I think this should be safe then. I will try it too (although I do have sensitive data :) ).

[jimmyhon]

Thanks @fberg. I moved the Portal 2 directory over onto a EXT4 partition, and the game runs fine. It was on a 2TB XFS partition with inode64 mount flags.

[Gijom]

Just to mention that the inode64.so solution worked for me too (although I have penty of warning "Cannot preload inode64.so").

[fberg]

Yeah, I get those warnings too, but they should only be due to the 64bit binaries which are executed along the way (e.g. bash since portal2.sh is a shell script). When running

$ LD_LIBRARY_PATH="./bin" fix32 ./portal2_linux -game portal2

directly from the Portal 2 directory, those warnings disappear for me. So the "cleanest" way would be to modify portal2.sh accordingly.

[cam72cam]

@fberg where does that fix32 command come from?

[Tele42]

@cam72cam fix32 is from the inodes64 link previously mentioned.

[cam72cam]

Ah, thank you!

[NTMan]

This issue affected not only Valve games. For example Brutal Legend is also affected and temporal solution is launch game with fix32 too: $ LD_LIBRARY_PATH=~/.local/share/Steam/ubuntu12_32/steam-runtime/i386/usr/lib/i386-linux-gnu/ fix32 ~/.local/share/Steam/steamapps/common/BrutalLegend/Buddha.bin.x86

I would like to see a normal solution, not crutches that worsen the stability of work.

[VortexAcherontic]

I have also observed the a similar behavior as @fberg https://github.com/ValveSoftware/portal2/issues/204#issuecomment-42778906 Portal 2 does not launch if it is not installed in the home directory Libaray folder.

I have an M2 as my home partition (not very much space obviously) and want to keep my games and other large applications on separate drives in separate Steam Libraries.

Also I witnessed Half-Life Half-Life 2 Are suffering form the same issue.

Running the games using Proton form a different Library works fine btw.

[CartoonFan]

Has there been any progress on this? I'm running into this issue with a bcachefs RAID setup. I have a 1 TB SSD used for caching, and a roughly 14TB HDD partition used for everything else.

EDIT: Forgot to mention that the 14TB partition is on the HDD.

[misyltoad]

I just tried to reproduce this on a 4TB ext4 drive and wasn't able to.

I guess the next step to know is whether the issue is caused by XFS/bcachefs or larger drives...

[CartoonFan]

Hey, @Joshua-Ashton! I forgot to mention that the 14TB partition's on my HDD, if that makes a difference.