build fails on AAch64, Fedora 33

[LutzWeischerFujitsu]

(base) [jw@cn06 build-it]$ ninja -v -j48 ... -- [download 100% complete] -- verifying file... file='/data/jw/apple/foundationdb/build-it/boost_targetProject-prefix/src/boost_1_72_0.tar.bz2' -- Downloading... done -- extracting... src='/data/jw/apple/foundationdb/build-it/boost_targetProject-prefix/src/boost_1_72_0.tar.bz2' dst='/data/jw/apple/foundationdb/build-it/boost_targetProject-prefix/src/boost_targetProject' -- extracting... [tar xfz] -- extracting... [analysis] -- extracting... [rename] -- extracting... [clean up] -- extracting... done [348/854] /usr/bin/cc -DBOOST_ERROR_CODE_HEADER_ONLY -DBOOST_SYSTEM_NO_DEPRECATED -DNO_INTELLISENSE -DSSD_ROCKSDB_EXPERIMENTAL -I../ -I. -O2 -DNDEBUG -DCMAKE_BUILD -ggdb -fno-omit-frame-pointer -Werror -Wno-pragmas -Wno-attributes -Wno-error=format -Wunused-variable -Wno-deprecated -fvisibility=hidden -Wreturn-type -fPIC -march=armv8.2-a+crc+simd -w -std=gnu11 -MD -MT fdbserver/CMakeFiles/fdb_sqlite.dir/sqlite/sqlite3.amalgamation.c.o -MF fdbserver/CMakeFiles/fdb_sqlite.dir/sqlite/sqlite3.amalgamation.c.o.d -o fdbserver/CMakeFiles/fdb_sqlite.dir/sqlite/sqlite3.amalgamation.c.o -c ../fdbserver/sqlite/sqlite3.amalgamation.c ninja: build stopped: subcommand failed. (base) [jw@cn06 build-it]$

[sfc-gh-anoyes]

Building with -j48 would almost certainly cause an OOM. Can you try building with lower parallelism? ~3Gb per process should be enough, but you might even need more memory.

[LutzWeischerFujitsu]

It even fails with -j4, about 275 steps are left to build. Is there no option to manage the process by limiting the RAM usage? I worked on hundreds of different softwares, but this is the most extreme case I ever came across. Why does it need so much RAM just for building the software? Thanks!

[sfc-gh-anoyes]

Why does it need so much RAM just for building the software?

Basically because of heavy use of c++ template metaprogramming. FWIW recent versions of clang seem better about memory usage.

It even fails with -j4

That makes me wonder if it isn't memory usage after all. Does it work with -j1? Is there anything indicating an actual compiler error (i.e. not the compiler process getting killed because of OOM) ? Can you monitor memory usage and see if it's actually running out?

[jzhou77]

@sfc-gh-kmakino do you know if ARM build fails or not?

[jzhou77]

BTW, ARM build is not fully supported yet, but it's on the roadmap.

[sfc-gh-kmakino]

I just built the master on aarch64 (AWS Graviton2) without any issues. One thing to note is that we're using foundationdb/build docker container which comes with CentOS7 + gcc8. We have not tested on other OSes or compilers.

[LutzWeischerFujitsu]

I'm having progress, e.g. a library was missing. A ran the test and get 2 errors:

(base) [jw@cn05 build-it]$ ninja -v -j$(nproc) test [0/1] cd /data/jw/apple/foundationdb/build-it && /usr/bin/ctest --force-new-ctest-process Test project /data/jw/apple/foundationdb/build-it Start 1: SimpleExternalTest ... 69/133 Test #69: restarting/from_7.0.0/ConfigureTestRestart-1 ....................Failed ... sec ... 90/133 Test #90: slow/DDBalanceAndRemoveStatus ...................................Failed ... sec ...

98% tests passed, 2 tests failed out of 133

Label Time Summary: ...

Total Test time (real) = ... sec

The following tests FAILED: 69 - restarting/from_7.0.0/ConfigureTestRestart-1 (Failed) 90 - slow/DDBalanceAndRemoveStatus (Failed) Errors while running CTest FAILED: CMakeFiles/test.util cd /data/jw/apple/foundationdb/build-it && /usr/bin/ctest --force-new-ctest-process ninja: build stopped: subcommand failed. (base) [jw@cn05 build-it]$

[williamdes]

Still accurate ?