How to enable caprevoke feature and build freeBSD with it?

[greenhandatsjtu]

I’m building freeBSD on my MacBook M2 (Ventura), I've checked out my freeBSD repo to dev branch following this issue: https://github.com/CTSRD-CHERI/cheribuild/issues/339

I can build and run freeBSD successfully by running:

./cheribuild.py --include-dependencies run-riscv64-purecap

Now I want to try caprevoke feature to avoid double free (if I don't misunderstand its usage), so I ran:

./cheribuild.py --include-dependencies --cheribsd/caprevoke-kernel run-riscv64-purecap

However, it failed with following errors:

bmake[1]: "/Users/sunhengke/cheri/cheribsd/share/mk/bsd.linker.mk" line 84: warning: /Users/sunhengke/cheri/cheribsd: Rerunning /Users/sunhengke/cheri/output/sdk/bin/ld.lld -v to compute LINKER_TYPE/LINKER_VERSION. This value should be cached!
bmake[1]: "/Users/sunhengke/cheri/cheribsd/Makefile.inc1" line 1802: Missing KERNCONF /Users/sunhengke/cheri/cheribsd/sys/riscv/conf/CHERI-CAPREVOKE-QEMU
bmake[1]: stopped in /Users/sunhengke/cheri/cheribsd
--- buildkernel ---
bmake: stopped in /Users/sunhengke/cheri/cheribsd
Fatal error (in target cheribsd-riscv64-purecap): Command `nice /Users/sunhengke/cheri/cheribsd/tools/build/make.py -j8 buildkernel TARGET=riscv TARGET_ARCH=riscv64c TARGET_CPUTYPE=cheri -DDB_FROM_SRC -DI_REALLY_MEAN_NO_CLEAN -DNO_ROOT -DBUILD_WITH_STRICT_TMPPATH 'LOCAL_XTOOL_DIRS=lib/libnetbsd lib/libsbuf usr.sbin/makefs usr.bin/mkimg' LD=/Users/sunhengke/cheri/output/sdk/bin/ld.lld XLD=/Users/sunhengke/cheri/output/sdk/bin/ld.lld AR=/Users/sunhengke/cheri/output/sdk/bin/llvm-ar NM=/Users/sunhengke/cheri/output/sdk/bin/llvm-nm OBJCOPY=/Users/sunhengke/cheri/output/sdk/bin/llvm-objcopy RANLIB=/Users/sunhengke/cheri/output/sdk/bin/llvm-ranlib SIZE=/Users/sunhengke/cheri/output/sdk/bin/llvm-size STRINGS=/Users/sunhengke/cheri/output/sdk/bin/llvm-strings STRIPBIN=/Users/sunhengke/cheri/output/sdk/bin/llvm-strip KERNCONF=CHERI-CAPREVOKE-QEMU DEBUG=-g -DWITHOUT_CLEAN -DWITH_TESTS -DWITH_CHERI -DWITHOUT_INIT_ALL_ZERO -DWITHOUT_INIT_ALL_PATTERN -DWITHOUT_MAN -DWITHOUT_MAIL -DWITHOUT_PROFILE -DWITHOUT_OFED -DWITH_MALLOC_PRODUCTION -DWITHOUT_GCC -DWITHOUT_CLANG -DWITHOUT_LLD -DWITHOUT_LLDB -DWITHOUT_GCC_BOOTSTRAP -DWITHOUT_CLANG_BOOTSTRAP -DWITHOUT_LLD_BOOTSTRAP -DWITHOUT_LIB32 -DWITH_ELFTOOLCHAIN_BOOTSTRAP -DWITH_TOOLCHAIN -DWITHOUT_BINUTILS_BOOTSTRAP -s -de` failed with non-zero exit code 2

Apparently, some config files are missing. Then I found a branch called caprevoke having these files, so I checked out to caprevoke and tried again. Sadly, it failed again with different errors:

clang: error: no such file or directory: '/Users/sunhengke/cheri/cheribsd/usr.sbin/config/main.cc'
clang: error: no input files
--- main.o ---
*** Failed target: main.o
*** Failed commands:
    ${CXX} ${STATIC_CXXFLAGS} ${CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET}
    => /usr/bin/clang++  -O2 -pipe -fno-common -I. -I/Users/sunhengke/cheri/cheribsd/usr.sbin/config -MD -MF.depend.main.o -MTmain.o -Wno-format-zero-length -Wsystem-headers -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Wdate-time -Wno-system-headers -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-error=unused-but-set-variable -Wno-typedef-redefinition -Werror=incompatible-pointer-types-discards-qualifiers -Qunused-arguments -I/Users/sunhengke/cheri/build/cheribsd-riscv64-purecap-build/Users/sunhengke/cheri/cheribsd/riscv.riscv64c/tmp/legacy/usr/include -Werror=implicit-function-declaration -Werror=implicit-int -Werror=return-type -Wundef -DHAVE_NBTOOL_CONFIG_H=1 -I/Users/sunhengke/cheri/cheribsd/tools/build/cross-build/include/common -D_DARWIN_C_SOURCE=1 -I/Users/sunhengke/cheri/cheribsd/tools/build/cross-build/include/mac -idirafter /Users/sunhengke/cheri/cheribsd/contrib/libarchive/libarchive     -Wno-c++11-extensions   -c /Users/sunhengke/cheri/cheribsd/usr.sbin/config/main.cc -o main.o
*** [main.o] Error code 1
bmake[3]: stopped in /Users/sunhengke/cheri/cheribsd/usr.sbin/config
1 error
bmake[3]: stopped in /Users/sunhengke/cheri/cheribsd/usr.sbin/config
--- _bootstrap-tools-usr.bin/awk ---
bmake[2]: stopped in /Users/sunhengke/cheri/cheribsd
--- _bootstrap-tools-lib/libelf ---
bmake[2]: stopped in /Users/sunhengke/cheri/cheribsd
--- _bootstrap-tools-kerberos5/lib/libroken ---
bmake[2]: stopped in /Users/sunhengke/cheri/cheribsd
--- _bootstrap-tools-usr.sbin/config ---
bmake[2]: stopped in /Users/sunhengke/cheri/cheribsd
--- _bootstrap-tools-lib/libdwarf ---
bmake[2]: stopped in /Users/sunhengke/cheri/cheribsd
--- _bootstrap-tools ---
bmake[1]: stopped in /Users/sunhengke/cheri/cheribsd
--- buildworld ---
bmake: stopped in /Users/sunhengke/cheri/cheribsd
Fatal error (in target cheribsd-riscv64-purecap): Command `nice /Users/sunhengke/cheri/cheribsd/tools/build/make.py -j8 buildworld TARGET=riscv TARGET_ARCH=riscv64c TARGET_CPUTYPE=cheri -DDB_FROM_SRC -DI_REALLY_MEAN_NO_CLEAN -DNO_ROOT -DBUILD_WITH_STRICT_TMPPATH 'LOCAL_XTOOL_DIRS=lib/libnetbsd lib/libsbuf usr.sbin/makefs usr.bin/mkimg' XAR=/Users/sunhengke/cheri/output/sdk/bin/llvm-ar XNM=/Users/sunhengke/cheri/output/sdk/bin/llvm-nm XSIZE=/Users/sunhengke/cheri/output/sdk/bin/llvm-size XSTRIPBIN=/Users/sunhengke/cheri/output/sdk/bin/llvm-strip XSTRINGS=/Users/sunhengke/cheri/output/sdk/bin/llvm-strings XOBJCOPY=/Users/sunhengke/cheri/output/sdk/bin/llvm-objcopy XRANLIB=/Users/sunhengke/cheri/output/sdk/bin/llvm-ranlib -DWITHOUT_CLEAN -DWITH_TESTS -DWITH_CHERI -DWITHOUT_INIT_ALL_ZERO -DWITHOUT_INIT_ALL_PATTERN -DWITHOUT_MAN -DWITHOUT_MAIL -DWITHOUT_PROFILE -DWITHOUT_OFED -DWITH_MALLOC_PRODUCTION -DWITHOUT_GCC -DWITHOUT_CLANG -DWITHOUT_LLD -DWITHOUT_LLDB -DWITHOUT_GCC_BOOTSTRAP -DWITHOUT_CLANG_BOOTSTRAP -DWITHOUT_LLD_BOOTSTRAP -DWITHOUT_LIB32 -DWITH_ELFTOOLCHAIN_BOOTSTRAP -DWITH_TOOLCHAIN -DWITHOUT_BINUTILS_BOOTSTRAP -s -de` failed with non-zero exit code 2

The question is, if I want to avoid double free (by using caprevoke feature), how can I build freeBSD? Thank you!

[jrtc27]

You'll need to do a clean build when switching branches like that.

[jrtc27]

if I want to avoid double free (by using caprevoke feature)

Double free is actually not really a concern of temporal safety and is easy for allocators to detect without the need for CHERI

[greenhandatsjtu]

You'll need to do a clean build when switching branches like that.

Do you mean I need to delete all cheribsd* directories under ~/cheri/build?

[greenhandatsjtu]

if I want to avoid double free (by using caprevoke feature)

Double free is actually not really a concern of temporal safety and is easy for allocators to detect without the need for CHERI

Sorry, actually what I want is exactly what you mentioned "temporal safety", which I learned from the papers has the ability to prevent temporal safety issues including UAF, UAR, and double free.

[jrtc27]

You'll need to do a clean build when switching branches like that.

Do you mean I need to delete all cheribsd* directories under ~/cheri/build?

--cheribsd/clean (or just --clean, which then applies to all targets you're building)

[greenhandatsjtu]

succeeded, thanks!