search

veracruz-project / veracruz

Main repository for the Veracruz privacy-preserving compute project, an adopted project of the Confidential Compute Consortium (CCC).
https://veracruz-project.com
MIT License
191 stars 39 forks source link

test-collateral should be able to create the artifacts needed to run a Veracruz instance without compiling all Veracruz #229

Open alexandref75 opened 3 years ago

alexandref75 commented 3 years ago

Requested feature A make target or instructions to create all the artifacts to run an instance of veracruz.

Motivation Reducing complexity for a user to try Veracruz and run and debug wasm applications. Some of the artifacts require to run Veracruz are not or can not (credentials, etc) provided into a ready to run environment (like a docker image). Instructions and/or make targets should be provided that construct those artifacts.

Additional context When trying to run make nitro-test-collateral

alexandr@alexandre-main ~/Projects/ARM/veracruz/veracruz $ make nitro-test-collateral TEE=nitro make -C test-collateral make[1]: Entering directory '/home/alexandr/Projects/ARM/veracruz/veracruz/test-collateral' make -C generate-policy make[2]: Entering directory '/home/alexandr/Projects/ARM/veracruz/veracruz/test-collateral/generate-policy' cargo build --release Updating crates.io index Updating git repository https://github.com/veracruz-project/ring.git Updating git repository https://github.com/veracruz-project/serde.git Updating git repository https://github.com/veracruz-project/json.git Updating git repository https://github.com/veracruz-project/wasi-types.git Updating git repository https://github.com/veracruz-project/rust-optee-trustzone-sdk.git Updating git repository https://github.com/veracruz-project/itoa.git Updating git repository https://github.com/veracruz-project/rustls.git Updating git repository https://github.com/veracruz-project/rust-base64.git Updating git repository https://github.com/veracruz-project/log.git Updating git repository https://github.com/veracruz-project/sct.rs.git Updating git repository https://github.com/veracruz-project/webpki.git Downloaded instant v0.1.11 Downloaded unicode-width v0.1.9 Downloaded tinyvec v1.5.0 Downloaded tracing v0.1.28 Downloaded syn v1.0.77 Downloaded libc v0.2.103 Downloaded 6 crates (910.3 KB) in 0.77s Compiling libc v0.2.103 Compiling proc-macro2 v1.0.29 Compiling unicode-xid v0.2.2 Compiling syn v1.0.77 Compiling autocfg v1.0.1 Compiling cfg-if v1.0.0 Compiling version_check v0.9.3 Compiling memchr v2.4.1 Compiling log v0.4.14 Compiling futures-core v0.3.17 Compiling proc-macro-hack v0.5.19 Compiling lazy_static v1.4.0 Compiling slab v0.4.4 Compiling cfg-if v0.1.10 Compiling futures-sink v0.3.17 Compiling futures-channel v0.3.17 Compiling futures-task v0.3.17 Compiling pin-project-lite v0.2.7 Compiling proc-macro-nested v0.1.7 Compiling bitflags v1.2.1 Compiling cc v1.0.70 Compiling futures-io v0.3.17 Compiling pin-utils v0.1.0 Compiling pin-project-lite v0.1.12 Compiling bytes v0.5.6 Compiling smallvec v1.6.1 Compiling pin-project-internal v0.4.28 Compiling ryu v1.0.5 Compiling parking_lot_core v0.8.5 Compiling autocfg v0.1.7 Compiling scopeguard v1.1.0 Compiling rand_core v0.4.2 Compiling getrandom v0.1.16 Compiling convert_case v0.4.0 Compiling tinyvec_macros v0.1.0 Compiling percent-encoding v2.1.0 Compiling matches v0.1.9 Compiling typenum v1.14.0 Compiling itoa v0.4.8 Compiling serde_derive v1.0.130 Compiling copyless v0.1.5 Compiling serde v1.0.130 Compiling bytes v1.1.0 Compiling unicode-segmentation v1.8.0 Compiling async-trait v0.1.51 Compiling unicode-bidi v0.3.6 Compiling either v1.6.1 Compiling ppv-lite86 v0.2.10 Compiling const_fn v0.4.8 Compiling match_cfg v0.1.0 Compiling fnv v1.0.7 Compiling adler v1.0.2 Compiling linked-hash-map v0.5.4 Compiling lexical-core v0.7.6 Compiling crc32fast v1.2.1 Compiling rustversion v1.0.5 Compiling regex-syntax v0.6.25 Compiling quick-error v1.2.3 Compiling serde_json v1.0.68 Compiling hashbrown v0.11.2 Compiling serde_derive v1.0.115 (https://github.com/veracruz-project/serde.git?branch=veracruz#19d2797f) Compiling encoding_rs v0.8.28 Compiling httparse v1.5.1 Compiling arrayvec v0.5.2 Compiling static_assertions v1.1.0 Compiling opaque-debug v0.3.0 Compiling untrusted v0.7.1 Compiling byteorder v1.4.3 Compiling spin v0.5.2 Compiling cpufeatures v0.2.1 Compiling serde v1.0.115 (https://github.com/veracruz-project/serde.git?branch=veracruz#19d2797f) Compiling base64 v0.13.0 Compiling log v0.4.8 (https://github.com/veracruz-project/log.git?branch=veracruz#64f2e089) Compiling gimli v0.25.0 Compiling failure_derive v0.1.8 Compiling mime v0.3.16 Compiling language-tags v0.2.2 Compiling rustc-demangle v0.1.21 Compiling base64 v0.11.0 Compiling base64 v0.10.1 (https://github.com/veracruz-project/rust-base64.git?branch=veracruz#9c3208ed) Compiling itoa v0.4.5 (https://github.com/veracruz-project/itoa.git?branch=veracruz#696e4c7c) Compiling unicode-width v0.1.9 Compiling strsim v0.8.0 Compiling hex v0.4.2 Compiling humantime v2.1.0 Compiling vec_map v0.8.2 Compiling termcolor v1.1.2 Compiling ansi_term v0.11.0 Compiling data-encoding v2.3.2 Compiling tracing-core v0.1.20 Compiling instant v0.1.11 Compiling lock_api v0.4.5 Compiling rand_core v0.3.1 Compiling rand_jitter v0.1.4 Compiling tinyvec v1.5.0 Compiling form_urlencoded v1.0.1 Compiling lru-cache v0.1.2 Compiling http v0.2.4 Compiling bytestring v1.0.0 Compiling heck v0.3.3 Compiling fxhash v0.2.1 Compiling textwrap v0.11.0 Compiling rand_chacha v0.1.1 Compiling rand_pcg v0.1.2 Compiling rand v0.6.5 Compiling futures-macro v0.3.17 Compiling futures-util v0.3.17 Compiling miniz_oxide v0.4.4 Compiling num-traits v0.2.14 Compiling num-integer v0.1.44 Compiling indexmap v1.7.0 Compiling num-bigint v0.2.6 Compiling rand_xorshift v0.1.1 Compiling rand_hc v0.1.0 Compiling rand_isaac v0.1.1 Compiling standback v0.2.17 Compiling generic-array v0.14.4 Compiling time v0.2.27 Compiling cookie v0.14.4 Compiling proc-macro-error-attr v1.0.4 Compiling nom v5.1.2 Compiling proc-macro-error v1.0.4 Compiling tracing v0.1.28 Compiling unicode-normalization v0.1.19 Compiling aho-corasick v0.7.18 Compiling object v0.26.2 Compiling quote v1.0.9 Compiling iovec v0.1.4 Compiling net2 v0.2.37 Compiling signal-hook-registry v1.4.0 Compiling num_cpus v1.13.0 Compiling hostname v0.3.1 Compiling rand_os v0.1.3 Compiling socket2 v0.3.19 Compiling time v0.1.44 Compiling atty v0.2.14 Compiling mio v0.6.23 Compiling flate2 v1.0.22 Compiling parking_lot v0.11.2 Compiling threadpool v1.8.1 Compiling addr2line v0.16.0 Compiling rand_core v0.5.1 Compiling regex v1.5.4 Compiling resolv-conf v0.7.0 Compiling idna v0.2.3 Compiling clap v2.33.3 Compiling err-derive v0.2.4 Compiling mio-uds v0.6.8 Compiling rand_chacha v0.2.2 Compiling block-buffer v0.9.0 Compiling digest v0.9.0 Compiling chrono v0.4.19 Compiling tokio v0.2.25 Compiling url v2.2.2 Compiling sha-1 v0.9.8 Compiling rand v0.7.3 Compiling env_logger v0.8.4 Compiling brotli-sys v0.3.2 Compiling ring v0.16.11 (https://github.com/veracruz-project/ring.git?branch=veracruz#84bc3ad5) Compiling backtrace v0.3.61 Compiling uuid v0.7.4 Compiling veracruz-utils v0.3.0 (/home/alexandr/Projects/ARM/veracruz/veracruz/veracruz-utils) Compiling rusticata-macros v2.1.0 Compiling der-parser v3.0.4 Compiling synstructure v0.12.5 Compiling x509-parser v0.7.0 Compiling tokio-util v0.3.1 Compiling sct v0.6.0 (https://github.com/veracruz-project/sct.rs.git?branch=veracruz#bb479fe4) Compiling webpki v0.21.2 (https://github.com/veracruz-project/webpki.git?branch=veracruz#0139cf73) Compiling rustls v0.16.0 (https://github.com/veracruz-project/rustls.git?branch=veracruz#39f39964) Compiling derive_more v0.99.16 Compiling actix-macros v0.1.3 Compiling thiserror-impl v1.0.29 Compiling pin-project-internal v1.0.8 Compiling enum-as-inner v0.3.3 Compiling time-macros-impl v0.1.2 Compiling proper v0.1.5 Compiling actix-web-codegen v0.4.0 Compiling time-macros v0.1.1 Compiling failure v0.1.8 Compiling brotli2 v0.3.2 Compiling pin-project v0.4.28 Compiling actix-codec v0.3.0 Compiling pin-project v1.0.8 Compiling tracing-futures v0.2.5 Compiling thiserror v1.0.29 Compiling actix-threadpool v0.3.3 Compiling actix-service v1.0.6 Compiling futures-executor v0.3.17 Compiling h2 v0.2.7 Compiling actix-rt v1.1.1 Compiling futures v0.3.17 Compiling trust-dns-proto v0.19.7 Compiling actix-utils v2.0.0 Compiling actix-server v1.0.4 Compiling actix-tls v2.0.0 Compiling trust-dns-resolver v0.19.7 Compiling actix-testing v1.0.1 Compiling actix-connect v2.0.0 Compiling serde_urlencoded v0.7.0 Compiling actix-router v0.2.7 Compiling actix-http v2.2.1 Compiling serde_json v1.0.51 (https://github.com/veracruz-project/json.git?branch=veracruz#30743088) Compiling wasi-types v0.1.6 (https://github.com/veracruz-project/wasi-types.git?branch=veracruz#c770d3d4) Compiling awc v2.0.3 Compiling actix-web v3.3.2 Compiling generate-policy v0.1.0 (/home/alexandr/Projects/ARM/veracruz/veracruz/test-collateral/generate-policy) Finished release [optimized] target(s) in 49.90s make[2]: Leaving directory '/home/alexandr/Projects/ARM/veracruz/veracruz/test-collateral/generate-policy' cp generate-policy/target/release/generate-policy ./pgen make css-nitro.bin -C ../runtime-manager make[2]: Entering directory '/home/alexandr/Projects/ARM/veracruz/veracruz/runtime-manager' Makefile:102: warning: overriding recipe for target 'runtime_manager.signed.so' Makefile:33: warning: ignoring old recipe for target 'runtime_manager.signed.so' Makefile:146: warning: overriding recipe for target 'e71bf7f6-702f-43e6-a897-cd7e1a231b06.ta' Makefile:143: warning: ignoring old recipe for target 'e71bf7f6-702f-43e6-a897-cd7e1a231b06.ta' make[2]: No rule to make target 'PCR0', needed by 'css-nitro.bin'. Stop. make[2]: Leaving directory '/home/alexandr/Projects/ARM/veracruz/veracruz/runtime-manager' make[1]: [Makefile:122: ../runtime-manager/css-nitro.bin] Error 2 make[1]: Leaving directory '/home/alexandr/Projects/ARM/veracruz/veracruz/test-collateral' make: *** [Makefile:57: nitro-test-collateral] Error 2

alexandref75 commented 3 years ago
Running the docker image (make nitro-exec) and (cd veracruz/test-collateral;TEE=nitro make) gets the following result: root@fbd0c74987fb:/work/veracruz/test-collateral# TEE=nitro make make -C generate-policy make[1]: Entering directory '/work/veracruz/test-collateral/generate-policy' cargo build --release Compiling http v0.2.5 Compiling futures-macro v0.3.17 Compiling futures-util v0.3.17 Compiling miniz_oxide v0.4.4 Compiling num-traits v0.2.14 error[E0658]: while is not allowed in a const fn --> /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/http-0.2.5/src/header/value.rs:85:9 85 / while i < bytes.len() { 86 if !is_visible_ascii(bytes[i]) { 87 ([] as [u8; 0])[0]; // Invalid header value 88 } 89 i += 1; 90 } _____^

= note: see issue #52000 https://github.com/rust-lang/rust/issues/52000 for more information = help: add #![feature(const_loop)] to the crate attributes to enable = help: add #![feature(const_if_match)] to the crate attributes to enable

error[E0658]: if is not allowed in a const fn --> /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/http-0.2.5/src/header/value.rs:86:13 86 / if !is_visible_ascii(bytes[i]) { 87 ([] as [u8; 0])[0]; // Invalid header value 88 } _____^

= note: see issue #49146 https://github.com/rust-lang/rust/issues/49146 for more information = help: add #![feature(const_if_match)] to the crate attributes to enable

Compiling num-integer v0.1.44 Compiling indexmap v1.7.0 Compiling num-bigint v0.2.6 Compiling standback v0.2.17 Compiling generic-array v0.14.4 Compiling time v0.2.27 Compiling cookie v0.14.4 Compiling proc-macro-error-attr v1.0.4 Compiling nom v5.1.2 Compiling proc-macro-error v1.0.4 Compiling memchr v2.4.1 error: aborting due to 2 previous errors

For more information about this error, try rustc --explain E0658. error: could not compile http.

To learn more, run the command again with --verbose. warning: build failed, waiting for other jobs to finish... error: build failed Makefile:15: recipe for target 'all' failed make[1]: [all] Error 101 make[1]: Leaving directory '/work/veracruz/test-collateral/generate-policy' Makefile:113: recipe for target 'pgen' failed make: [pgen] Error 2 root@fbd0c74987fb:/work/veracruz/test-collateral#

dominic-mulligan-arm commented 3 years ago

To be clear, "all of Veracruz" isn't being compiled here: rather, only the Rust program that generates the policy files is being compiled, which seems reasonable. One way around this problem, for demo purposes, is to have a fixed, pre-built policy file for use with the demo.

The second problem is a legitimate bug caused by a dependency silently updating and becoming incompatible with our pinned version of the Rust compiler.

alexandref75 commented 3 years ago

I may be doing something wrong here but AFAIK runtime-manager is not part of the test-collateral, so it may not be all of veracruz but it seems a very sizable part of it.

~/Projects/ARM/veracruz/veracruz/test-collateral $ TEE=nitro make make -C generate-policy make[1]: Entering directory '/home/alexandr/Projects/ARM/veracruz/veracruz/test-collateral/generate-policy' cargo build --release Finished release [optimized] target(s) in 1.16s make[1]: Leaving directory '/home/alexandr/Projects/ARM/veracruz/veracruz/test-collateral/generate-policy' cp generate-policy/target/release/generate-policy ./pgen make css-nitro.bin -C ../runtime-manager make[1]: Entering directory '/home/alexandr/Projects/ARM/veracruz/veracruz/runtime-manager' Makefile:102: warning: overriding recipe for target 'runtime_manager.signed.so' Makefile:33: warning: ignoring old recipe for target 'runtime_manager.signed.so' Makefile:146: warning: overriding recipe for target 'e71bf7f6-702f-43e6-a897-cd7e1a231b06.ta' Makefile:143: warning: ignoring old recipe for target 'e71bf7f6-702f-43e6-a897-cd7e1a231b06.ta' make[1]: No rule to make target 'PCR0', needed by 'css-nitro.bin'. Stop. make[1]: Leaving directory '/home/alexandr/Projects/ARM/veracruz/veracruz/runtime-manager' make: [Makefile:130: ../runtime-manager/css-nitro.bin] Error 2

alexandref75 commented 3 years ago

The second problem, artifact necessary, is related to the proxy-attestation-server.db that is not a policy and is needed for the proxy-attestation-server to operate. The documentation describes this file as required to be rebuilt every time the executable of the enclave is recreated. This seems only to be necessary for SGX but the proxy-attestation-server for nitro will not operate without the database.