tsinghua-ideal / flare

Revise Spark written in Rust (Native Spark or Vega) to a secure one
9 stars 1 forks source link

Error: `Option::unwrap()` on a `None` value' #6

Open fantas1a opened 2 days ago

fantas1a commented 2 days ago

Hi, I wonder if anyone can help with the following errors. I run SODA with one master and two slave nodes, and all with scalable SGX.

name@master:~/soda/bin$ sudo ./app 15:34:24 [ INFO] starting local_fs_reader<bytes> job thread_id = 139655171670016 finish pre_touching local_scheduler serialize task time: 0.002882157 s local_scheduler deserialize task time: 7.3917e-5 s tid: 11, at the begining of secure execution thread '' panicked at 'called Option::unwrap() on a None value', src/lib.rs:244:72 Illegal instruction

Here is my node configuration:

master = "10.0.0.5:3000" slaves = ["user@10.0.0.6", "user@10.0.0.7"]

My machine:

ubuntu 20.04 with SGX v2, every nodes can ssh each other without password

And following is what I did:

  1. Install rustup from its website by command: $ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
  2. Install 2.14 version SGX (sdk, psw, driver)
  3. Run the first 2 commands from spark vega (https://rajasekarv.github.io/vega/): $ rustup toolchain install nightly $ rustup default nightly
  4. Install Cap'n Proto by: $ apt-get install capnproto
  5. Paste Cargo.lock from this issue (https://github.com/tsinghua-ideal/flare/issues/3)
  6. Compile by type: $ make
  7. It prompts rustfmt is not available in this nightly version, suggest if I do not need rustfmt, I can use minimal installation. Otherwise, use the most update rustup this year. I use the suggested commands to use minimal installation.

It prompted: info: syncing channel updates for 'nightly-2021-03-30-x86_64-unknown-linux-gnu' info: latest update on 2021-03-30, rust version 1.53.0-nightly (48691ea6e 2021-03-29) error: component 'rustfmt' for target 'x86_64-unknown-linux-gnu' is unavailable for download for channel 'nightly-2021-03-30' Sometimes not all components are available in any given nightly. If you don't need the component, you could try a minimal installation with:

rustup toolchain add nightly-2021-03-30 --profile minimal

If you require these components, please install and use the latest successful build version, which you can find at https://rust-lang.github.io/rustup-components-history.

After determining the correct date, install it with a command such as:

rustup toolchain install nightly-2018-12-27

Then you can use the toolchain with commands such as:

cargo +nightly-2018-12-27 build

make: *** [Makefile:279: clean] Error 1

I type in:

rustup toolchain add nightly-2021-03-30 --profile minimal
  1. Compile again by type: $ make
  2. Make succeed. Copy the same hosts.conf file to every nodes at their /root/
  3. Run on every nodes: $ source ~/flare/bin/set_env_dist.sh
  4. Run on master node: $ sudo ~/flare/bin/app
fantas1a commented 1 day ago

I have same error when in local mode:

05:19:43 [ INFO] starting local_fs_reader<bytes> job thread_id = 140685963821056 finish pre_touching local_scheduler serialize task time: 0.0030098890000000004 s local_scheduler deserialize task time: 8.0407e-5 s tid: 11, at the begining of secure execution thread '' panicked at 'called Option::unwrap() on a None value', src/lib.rs:244:72 Illegal instruction

And the hosts.conf file I used when in local mode:

master = "10.0.0.6:3000" slaves = [""]

I executed the ~/flare/bin/set_env_distri.sh in advance.

fantas1a commented 1 day ago

@AmbitionXiang Hearty thanks if you could take a look at this : )

fantas1a commented 1 day ago

And if I test with se1_sec()?; se2_sec()?; se3_sec()?; in app/src/main.rs, it prompted:

09:56:41 [ INFO] starting unknown job thread_id = 139655171670016 finish pre_touching local_scheduler serialize task time: 0.0030862180000000004 s local_scheduler serialize task time: 4.7784e-5 s local_scheduler deserialize task time: 0.00013935800000000002 s in denepdency, key = (16961387396503108670, 0, 0), ops = [OpId { h: 14338286665493845171 }, OpId { h: 17013284107314768422 }, OpId { h: 17639518093728688273 }, OpId { h: 10378376576142600703 }, OpId { h: 11054832435461403850 }] thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: InputRead(Os { code: 2, kind: NotFound, message: "No such file or directory" })', /home/dbucket/flare/flare-core-untrusted/src/dependency.rs:294:74 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace local_scheduler deserialize task time: 5.2827e-5 s in denepdency, key = (15998552245887240225, 0, 1), ops = [OpId { h: 14338286665493845171 }, OpId { h: 7045621555540763815 }, OpId { h: 8622552163589584583 }, OpId { h: 12031491541542741121 }, OpId { h: 10809193829318009398 }] thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: InputRead(Os { code: 2, kind: NotFound, message: "No such file or directory" })', /home/dbucket/flare/flare-core-untrusted/src/dependency.rs:294:74

Here is the main function I used:

fn main() -> Result<()> { //Fn! will make the closures serializable. It is necessary. use serde_closure version 0.1.3. / dijkstra / //dijkstra_sec_0()?; //dijkstra_unsec_0()?;

/* map */
//map_sec_0()?;
//map_sec_1()?;
//map_unsec_0()?;
//map_unsec_1()?;

/* filter */
//filter_sec_0()?;
//filter_unsec_0()?;

/* group_by */
//group_by_sec_0()?;
//group_by_sec_1()?;

/* join */
//join_sec_0()?;
//join_sec_1()?;
//join_sec_2()?;
//join_unsec_2()?;

/* distinct */
//distinct_sec_0()?;
//distinct_unsec_0()?;

/* local file reader */
//file_read_sec_0()?;
//file_read_unsec_0()?;

/* partition_wise_sample */
//part_wise_sample_sec_0()?;
//part_wise_sample_unsec_0()?;
//part_wise_sample_unsec_1()?;
//part_wise_sample_unsec_2()?;

/* take */
//take_sec_0()?;
//take_unsec_0()?;

/* reduce */
//reduce_sec_0()?;

/* count */
//count_sec_0()?;
//count_unsec_0()?;

/* union */
//union_sec_0()?;
//union_unsec_0()?;

/* zip */
//zip_sec_0()?;

/* big data bench */
// aggregate_sec()?;
// filter_sec()?;
// cross_project_sec()?;
// aggregate_unsec()?;
// filter_unsec()?;
// cross_project_unsec()?;

/* tpc-h */
// te1_sec()?;
// te2_sec()?;
// te3_sec()?;
// te1_unsec()?;
// te2_unsec()?;
// te3_unsec()?;

/* social graph */
 se1_sec()?;
 se2_sec()?;
 se3_sec()?;
// se1_unsec()?;
// se2_unsec()?;
// se3_unsec()?;

/* kmeans */
//kmeans_sec_0()?;
//kmeans_unsec_0()?;

/* linear regression */
//lr_sec()?;
//lr_unsec()?;

/* matrix multipilication */
//mm_sec_0()?;
//mm_unsec_0()?;

/* page rank */
//pagerank_sec_0()?;
//pagerank_unsec_0()?;

/* pearson correlation algorithm */
//pearson_sec_0()?;
//pearson_unsec_0()?;

/* transitive_closure */
//transitive_closure_sec_0()?;
//transitive_closure_sec_1()?;
//transitive_closure_unsec_0()?;
//transitive_closure_unsec_1()?;

/* triangle counting */
//triangle_counting_sec_0()?;
//triangle_counting_unsec_0()?;

// test the speculative execution in loop
//test0_sec_0()?;

//topk
//topk_sec_0()?;
//topk_unsec_0()?;

Ok(())

}