facebook / mvfst

An implementation of the QUIC transport protocol.
MIT License
1.5k stars 242 forks source link

The build fills up /tmp folder and fails #283

Closed alexj0l closed 5 months ago

alexj0l commented 1 year ago

Hi community,

Before the build with ./getdeps.sh I had 11G available space in /tmp folder but the build fills everything up and then crashes with the error of 'No space left on device'. Can someone help me with that (maybe disable some space-eating features of mvfst?)? My goal is to build mvfst and use the tperf to test it's throughput.

Thanks in advance!

lnicco commented 1 year ago

This is partly due to the getdeps build and partly due to mvfst specific config.

The getdeps build system by default recompiles all the dependencies for more reproducible builds.

One small optimization is to use system dependencies where possible by running

./getdeps.sh --allow-system-packages

Which would save a few GB

ubuntu@foo.bar:~/lnicco/mvfst$ du -sh /tmp/fbcode_builder_getdeps-ZhomeZubuntuZ*
16G     /tmp/fbcode_builder_getdeps-ZhomeZubuntuZlniccoZmvfstZbuildZfbcode_builder
13G     /tmp/fbcode_builder_getdeps-ZhomeZubuntuZlniccoZmvfstZbuildZfbcode_builder.with_system_deps

The other reason this is using a lot of disk space is that the binaries (tests included) all have debug symbols by default

Using the following command should remove the debug symbols:

./getdeps.sh --extra-cmake-defines='{ "CMAKE_BUILD_TYPE": "Release" }'
...
ubuntu@foo.bar:~/lnicco/mvfst$ du -sh /tmp/fbcode_builder_getdeps-ZhomeZubuntuZlniccoZmvfstZbuildZfbcode_builder
7.1G    /tmp/fbcode_builder_getdeps-ZhomeZubuntuZlniccoZmvfstZbuildZfbcode_builder

What we can do is Make the Release build type the default.

If you are low on disk space you can also add the following lines to the main CMake file, to make sure that all the binaries get stripped when building in Release mode.

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 8673c3f0..9340d44f 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -11,6 +11,9 @@ set(CMAKE_CXX_STANDARD 17)
 set(CMAKE_CXX_STANDARD_REQUIRED ON)
 set(CMAKE_CXX_EXTENSIONS OFF)

+set(CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} -s")
+set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -s")
+
 set(CMAKE_MODULE_PATH
   "${CMAKE_CURRENT_SOURCE_DIR}/cmake"
   # for in-fbsource builds
alexj0l commented 1 year ago

Will this affect the throughput I will see using tperf?

jbeshay commented 1 year ago

Using the system packages and stripping the debug symbols should not have impact on tperf throughput. Although stripping the debug symbols could make it harder for you to debug issues, especially if you're making changes to the code.