Max: A Multicore-Accelerated File System for Flash Storage

Max is a file system designed for modern multicore CPUs and high-performance SSDs. This is the source code repository of Max.

Please read the paper "Max: A Multicore-Accelerated File System for Flash Storage" of USENIX ATC'21 for more details.

If you use this work, please cite:

@inproceedings {273904,
author = {Xiaojian Liao and Youyou Lu and Erci Xu and Jiwu Shu},
title = {Max: A Multicore-Accelerated File System for Flash Storage},
booktitle = {2021 {USENIX} Annual Technical Conference ({USENIX} {ATC} 21)},
year = {2021},
isbn = {978-1-939133-23-6},
pages = {877--891},
url = {https://www.usenix.org/conference/atc21/presentation/liao},
publisher = {{USENIX} Association},
month = jul,
}

WARNING: The code can be only used for academic research. Please do not use it for production.

Contact information: liaoxiaojian@tsinghua.edu.cn

Downloading the source code

apt install git-lfs
git lfs clone git@github.com:thustorage/max.git

Building Max (be root user)

1. Build the Linux kernel

   apt install -y flex bison make gcc libssl-dev bc libelf-dev
   tar -xf linux-kernel.tar.gz  
   cd linux-kernel
   cp /boot/config-`uname -r` .config
   make olddefconfig
   make -j24 && make modules_install -j24 && make install

2. Update the grub configuration and reboot into the new kernel.
3. Build and install Max

   cd Max
   make -j24
   insmod max.ko

4. Build the disk format tool

   cd disk-tools
   apt install pkg-config autoconf libtool libselinux1-dev
   ./autogen.sh
   ./configure
   make -j24

Running Max

1. Format the drive

   disk-tools/mkfs/mkfs.f2fs -N 8 /dev/nvme0n1

   Note: the number 8 means the maximum number of mlogs. You can replace it with a proper number lower than 64.

2. Mount the drive

   mount -t max -o imds=72,mlog=8 /dev/nvme0n1 /mnt/test

   Note: the number of 72 means the number of file cell groups. the number of 8 means the number of mlogs.

Now, the Max file system is mounted at /mnt/test, storing its data on /dev/nvme0n1.

Testing Max with FxMark

1. Build FxMark

   cd fxmark
   make -j24

2. Build Exim

   cd vbench/exim
   make -j24

3. Build RocksDB

   cd vbench/rocksdb
   make -j24

4. Setup the test. Open the file bin/run-fxmark.py.
   • Setup the tested devices in lines 30-33. An example:

     HDDDEV = "/dev/sda1"
     SSDDEV = "/dev/sdb1"
     NVMEDEV = "/dev/nvme0n1"
     LOOPDEV = "/dev/loop0"

   • Setup the tested media and file systems in lines 61-75. An example:

     self.MEDIA_TYPES = ["ssd","hdd","nvme",]
     self.FS_TYPES = ["ext4","xfs","f2fs","max",]   

   • Setup the workloads in lines 78-127. An example:

     self.BENCH_TYPES = ["DWOL","DWAL","MWCL","MWUL","filebench_varmail","rocksdb_overwrite"]

   • Setup the test in lines 632-635. An example:

     run_config = [(Runner.CORE_FINE_GRAIN, PerfMon.LEVEL_LOW,("nvme","max","DWOL","72","bufferedio"), "", "")]

     Note: the entry "nvme" specifies the medium. The entry "max" specifies the file system. The entry "DWOL" (data overwrite) specifies the workload. The entry "72" specifies the number of cores. You can set the entries to "*" to enable all media, file systems, workloads and numbers of cores.

5. Run the tests.

   bin/run-fxmark.py