help needed porting substrate-api-client to sgx

[brenzi]

Hi @dingelish. We wrote a crate that needs to be used within an enclave now. Our effort to port can be found here: https://github.com/scs/substrate-api-client/tree/no_std

The current master works fine. The no_std branch fails at:

> cargo build --release 
   Compiling substrate-api-client v2.0.0 (/home/abrenzikofer/substrate-api-client)
error[E0432]: unresolved import `std::sync::mpsc`
  --> src/lib.rs:32:16
   |
32 | use std::sync::mpsc::channel;
   |                ^^^^ help: a similar path exists: `ws::std::sync::mpsc`

as well as at

cargo build --release --no-default-features --features mesalock_sgx
   Compiling substrate-api-client v2.0.0 (/home/abrenzikofer/substrate-api-client)
error[E0432]: unresolved import `std::vec`
  --> src/extrinsic/mod.rs:22:10
   |
22 | use std::vec::Vec;
   |          ^^^ help: a similar path exists: `serde::std::vec`

as well as without the mesalock_sgx feature.

Your help would be greatly appreciated

[tomtau]

@brenzi just a quick look: https://github.com/scs/substrate-api-client/blob/no_std/Cargo.toml#L119

https://github.com/baidu/rust-sgx-sdk/tree/master/sgx_tstd/src/sync sgx_tstd doesn't implement all of std:

https://github.com/baidu/rust-sgx-sdk/blob/master/release_notes.md#partially-supported-modstraits-in-sgx_tstd

[brenzi]

Thanks @tomtau. I'm aware that sgx_tstd only implements std partially. I think I've gated off everything which isn't covered by #[cfg(feature = "std")] https://github.com/scs/substrate-api-client/blob/1dad3a835d8f2f796c43cfcb4f9ae718bef44cc4/src/lib.rs#L31

[tomtau]

Yes, butstd is on in cargo build --release.

for cargo build --release --no-default-features --features mesalock_sgx, perhaps use std::prelude::v1::Vec; instead of use std::vec::Vec; ?

[brenzi]

I suspect the problem to be with the Cargo.toml file. I could fix some of the above errors by adding a few default-features=false to mesalock deps.

If I do cargo build --release, I'd expect sgx_tstd not to be a dependency, but here's my cargo tree:

sgx_tstd v1.0.9 (https://github.com/baidu/rust-sgx-sdk?rev=v1.0.9#407cbaaf)
├── itoa v0.4.4 (https://github.com/mesalock-linux/itoa-sgx.git#4f4b8dc2)
│   └── serde_json v1.0.40 (https://github.com/mesalock-linux/serde-json-sgx#7f602959)
│       └── substrate-api-client v2.0.0 (/home/abrenzikofer/substrate-api-client)
├── log v0.4.8 (https://github.com/mesalock-linux/log-sgx#93cf0f64)
│   ├── env_logger v0.6.2 (https://github.com/mesalock-linux/env_logger-sgx#b2f44420)
│   │   └── substrate-api-client v2.0.0 (/home/abrenzikofer/substrate-api-client) (*)
│   └── substrate-api-client v2.0.0 (/home/abrenzikofer/substrate-api-client) (*)
└── serde v1.0.101 (https://github.com/mesalock-linux/serde-sgx#bc160eaa)
    ├── serde_json v1.0.40 (https://github.com/mesalock-linux/serde-json-sgx#7f602959) (*)
    └── substrate-api-client v2.0.0 (/home/abrenzikofer/substrate-api-client) (*)

Looking at serde-json-sgx, I see that the std feature is not forwarded to dependency itoa. I guess this should be added, no? update here: https://github.com/scs/substrate-api-client/tree/no_std

[brenzi]

It seems that serde_json-sgx depends on itoa without default-features=false https://github.com/mesalock-linux/serde-json-sgx/blob/7f6029590cf312c412c225cb944c429e5fadd3b2/Cargo.toml#L22

which is translated to default=["mesalock-sgx"] here:

https://github.com/mesalock-linux/itoa-sgx/blob/4f4b8dc29204fc7408584df7083194329aa5cc59/Cargo.toml#L14

which then drags in sgx_tstd which is undesired

[brenzi]

@dingelish : AFAIK you have no samplecode that implements a crate that can be used within an enclave AS WELL AS outside in std environment. To my understanding, that's also why you have split sgx_crypto_helper and sgx_tcrypto_helper.

I'm trying a wrong approach? Should I just support no_std without relying on your sgx_tstd libs?

[dingelish]

@dingelish : AFAIK you have no samplecode that implements a crate that can be used within an enclave AS WELL AS outside in std environment. To my understanding, that's also why you have split sgx_crypto_helper and sgx_tcrypto_helper.

I'm trying a wrong approach? Should I just support no_std without relying on your sgx_tstd libs?

Almost all of the forked crates are only available in enclave. But there are exceptions like serde, which has almost zero dependencies and I remember you can enable "std" without "mesalock_sgx" to make it work in normal untrusted app.

And you're right about crypto_helper and tcrypto_helper.

I would say that no_std crate is friendly to SGX enclave, but it sacrifices many stuffs. Let me look closer to your case and reply later :-)

[dingelish]

Hi @brenzi , do you prefer the branch to be compiled enclave-only, or prefer to make it work on both untrusted side and trusted side?

[brenzi]

untrusted AND trusted with the same crate.

Right now my understanding is that this is not really supported by mesalock forks. So I consider doing it parity style, using https://github.com/paritytech/substrate/tree/master/core/sr-std. (another replacement for std, similar to sgx-tstd but even smaller (less coverage). I know it works within enclaves

What do you think would the best way?

[brenzi]

One more remark: I only need parts of the substrate-api-client crate to be available on trusted side. See lib.rs for exclusions with #[cfg(feature = "std")]

All I really need to work within enclaves is the following macro: https://github.com/scs/substrate-api-client/blob/54394fec90297ea74729cd29e4a82425a31d5737/src/extrinsic/mod.rs#L64

[brenzi]

I think the parity style can work: https://github.com/scs/substrate-api-client/tree/no_std_rstd

at least it's building. further testing ahead.