Writing traces to anything but tmpfs causes hickups in gameplay

[timjp87]

Hello,

when creating traces on my HDD the gameplay stops for about half a second every 10 seconds or so.

Tracing to the SSD shows the same issue. When replaying these traces the hickups are not visible in the replay. I monitored my system resources and did not notice anything unusual with RAM and CPU usage.

When I point the output file to /tmp I can play without hickups, but since its size is only half my RAM the trace aborts when /tmp is full.

https://gist.github.com/blackout24/9655278

I was able to reproduce this with two games (Sauerbraten and Xonotic).

[richgel999]

Can you profile the traced app to see were the bottleneck really is? Tracing to an SSD on either of my systems is pretty good, but I get hickups when tracing to a regular HD.

I'm guessing it's caused by the kernel blocking us until it has flushed enough file output to disk. We do very little compute in the tracer (just compose packets, memcpy and CRC some stuff, and issue 1 fwrite() per GL call). If that's the case, lossless trace compression on a helper thread will help somewhat.

On Wed, Mar 19, 2014 at 6:21 PM, blackout24 notifications@github.comwrote:

Hello,

when creating traces on my HDD the gameplay stops for about half a second every 10 seconds or so.

Tracing to the SSD shows the same issue. When replaying these traces the hickups are not visible in the replay. I monitored my system resources and did not notice anything unusual with RAM and CPU usage.

When I point the output file to /tmp I can play without hickups, but since its size is only half my RAM the trace aborts when /tmp is full.

https://gist.github.com/blackout24/9655278

I was able to reproduce this with two games (Sauerbraten and Xonotic).

Reply to this email directly or view it on GitHubhttps://github.com/ValveSoftware/vogl/issues/29 .

[timjp87]

Which profiling tools would be useful?

I followed this tutorial for system wide profiling with perf and recorded 35 seconds of activity. http://www.pixelbeat.org/programming/profiling/

This is the perf.data when capturing to tmpfs: https://www.dropbox.com/s/bivfn7hq3mk44ha/perf_tmp.data I had no hickups during gameplay.

This is the perf.data when capturing to the HDD: https://www.dropbox.com/s/gpw14pbbzus6577/perf_hdd.data I had multiple and long hickups during gameplay.

This is the perf.data when capturing to the SSD: https://www.dropbox.com/s/5ix7zj8ejjhfyni/perf_ssd.data I had one brief hickup during gameplay.

[richgel999]

Yes, this data should be useful. I'll check this out in a bit.

Sent from my iPad

On Mar 20, 2014, at 7:48 AM, blackout24 notifications@github.com wrote:

Which profiling tools would be useful?

I followed this tutorial for system wide profiling with perf and recorded 35 seconds of activity.: http://www.pixelbeat.org/programming/profiling/

This is the perf.data when capturing to tmpfs: https://www.dropbox.com/s/bivfn7hq3mk44ha/perf_tmp.data I had no hickups during gameplay.

This is the perf.data when capturing to the HDD: https://www.dropbox.com/s/gpw14pbbzus6577/perf_hdd.data I had multiple and long hickups during gameplay.

This is the perf.data when capturing to the SSD: https://www.dropbox.com/s/5ix7zj8ejjhfyni/perf_ssd.data I had one brief hickup during gameplay.

— Reply to this email directly or view it on GitHub.

[amonakov]

@blackout24, it's odd that it blocks that badly, but the issue seems tied to disk writeback. Can you try tuning page cache writeback according to http://lonesysadmin.net/2013/12/22/better-linux-disk-caching-performance-vm-dirty_ratio/ ?

[timjp87]

@amonakov The thrid approach seems to have some positiv effects, but it's hard to tell since there are still noticable hickups from time to time. This is my subjective feeling.

[amonakov]

How much RAM do you have? Try making background writeback more eager: sudo sysctl vm.dirty_background_bytes=$[2**21]

[timjp87]

I have 8 GB RAM with free -m teeling me that there is about 7 GB free before starting the capture.

vm.dirty_background_bytes=$[2**21] improved things dramatically, while having everything else at the default values.

My captures always ranged between 2-5 GB with all the games I have tested. Now I'm able to capture 1 minute of Xonotic to HDD without a single hickup. The resulting capture is 5.2 GB. I tried the same with the three approaches outlined in the tutorial and basically with all of them the game was more or less unplayable during the same 1 minute timeframe.

I just captured 17 GB to HDD and only had a few very brief pauses with very long periods of fluid gameplay between them.