This is the secure version of Native Spark
, or Vega
(https://github.com/rajasekarv/vega). It contains two modes: encryption mode and oblivious mode. You can switch the mode by switching to the expected branch.
It contains two submodules named flare-core-untrusted
, which is the focus to modify, and incubator-teaclave-sgx-sdk
, the Rust SGX SDK developed by Baidu X Lab. flare-core-untrusted
has one branch master
, which is modified by me based on master
branch of Vega
repo.
Please refer to https://www.rust-lang.org/tools/install. It's recommended to use rustup
for management.
We require Intel SGX SDK 2.14.
Tasks will not run on the master node, and all the tasks run on worker nodes. So, SGX toolchain (Driver, PSW, SDK) needs to be installed on worker nodes, while SDK needs to be installed on the leader node for compiling the code. All are installed in /opt/intel/
.
Checkout the secure
branch of native spark
first.
Please refer to https://rajasekarv.github.io/vega/ for Getting started
part and Executing an application
part.
Just type make
If you use this framework in your research, please kindly cite the following papers.
Xiang Li, Fabing Li, and Mingyu Gao. Flare: A Fast, Secure, and Memory-Efficient Distributed Analytics Framework. Proc. VLDB Endow. 16, 6 (2023), 1439–1452.
Xiang Li, Nuozhou Sun, Yunqian Luo, and Mingyu Gao. SODA: A Set of Fast Oblivious Algorithms in Distributed Secure Data Analytics. Proc. VLDB Endow. 16, 7 (2023), 1671-1684.