search

openethereum / parity-ethereum

The fast, light, and robust client for Ethereum-like networks.
Other
6.83k stars 1.69k forks source link

Error building Open Ethereum for MIPS using Cross on macOS `failed to run custom build command for ring v0.16.9` #11549

Open ltfschoen opened 4 years ago

ltfschoen commented 4 years ago

expected

I was expecting (hoping) it would compile successfully.

actual

I've attempted to try to use Linux in a Docker Container to build Open Ethereum to MIPS (big endian hardware) using Cross, however I'm currently encountered this issue https://github.com/docker/for-linux/issues/951, which is a progression from my initial issue https://github.com/rust-embedded/cross/issues/385. All the code and detailed steps that I've taken are included in this WIP PR in my fork https://github.com/ltfschoen/open-ethereum/pull/1.

So whilst I was waiting for help I've tried to just try building Open Ethereum to MIPS using Cross directly on my macOS local machine.

However, after installing relevant dependencies for Parity Ethereum on macOS by running my script dependencies_mac.sh,

Then cloning Open Ethereum with:

git clone https://github.com/OpenEthereum/open-ethereum;
cd open-ethereum;
rustup default nightly;
git checkout master;
cargo clean;

Then usually I'd run cargo build --release --features final and it would build successfully.

However I want to use cross https://github.com/rust-embedded/cross (instead of cargo) to build to a MIPS (big endian hardware) release target (instead of little endian hardware), but I get the error shown below:

ls @ ~/code/open-ethereum - [luke/poa-tutorial] $ RUST_BACKTRACE=1 QEMU_STRACE=1 cross run --target mips-unknown-linux-gnu --release --features final
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: latest update on 2020-03-06, rust version 1.43.0-nightly (96bb8b31c 2020-03-05)
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-docs'
info: downloading component 'rust-std'
info: downloading component 'rustc'
 60.2 MiB /  60.2 MiB (100 %)   2.3 MiB/s in 16s ETA:  0s
info: downloading component 'rustfmt'
  3.4 MiB /   3.4 MiB (100 %)   3.2 MiB/s in  1s ETA:  0s
info: installing component 'cargo'
info: installing component 'clippy'
info: installing component 'rust-docs'
 12.2 MiB /  12.2 MiB (100 %)   2.9 MiB/s in  3s ETA:  0s
info: installing component 'rust-std'
info: installing component 'rustc'
 60.2 MiB /  60.2 MiB (100 %)  15.0 MiB/s in  4s ETA:  0s
info: installing component 'rustfmt'

  nightly-x86_64-unknown-linux-gnu installed - (error reading rustc version)

info: checking for self-updates
info: downloading component 'rust-std' for 'mips-unknown-linux-gnu'
 15.8 MiB /  15.8 MiB (100 %)   3.7 MiB/s in  4s ETA:  0s
info: installing component 'rust-std' for 'mips-unknown-linux-gnu'
Unable to find image 'rustembedded/cross:mips-unknown-linux-gnu-0.2.0' locally
mips-unknown-linux-gnu-0.2.0: Pulling from rustembedded/cross
Digest: sha256:591cd3609de9f3c40db6e73456679674a7b87988a0811ebfc3f72c736f37ecc1
Status: Downloaded newer image for rustembedded/cross:mips-unknown-linux-gnu-0.2.0
  Downloaded xdg v2.1.0
   Compiling libc v0.2.65
   Compiling proc-macro2 v1.0.8
   Compiling unicode-xid v0.2.0
   Compiling syn v1.0.14
   Compiling serde v1.0.103
   Compiling semver-parser v0.7.0
   Compiling cfg-if v0.1.10
   Compiling autocfg v1.0.0
   Compiling byteorder v1.3.2
   Compiling lazy_static v1.4.0
   Compiling maybe-uninit v2.0.0
   Compiling rayon-core v1.7.0
   Compiling scopeguard v1.0.0
   Compiling either v1.5.0
   Compiling log v0.4.8
   Compiling getrandom v0.1.11
   Compiling autocfg v0.1.7
   Compiling crunchy v0.2.2
   Compiling ppv-lite86 v0.2.5
   Compiling rustc-hex v2.1.0
   Compiling rand_core v0.4.2
   Compiling futures v0.1.29
   Compiling static_assertions v1.1.0
   Compiling bitvec v0.15.2
   Compiling byte-slice-cast v0.3.4
   Compiling arrayvec v0.5.1
   Compiling smallvec v1.2.0
   Compiling ryu v1.0.0
   Compiling slab v0.4.1
   Compiling itoa v0.4.3
   Compiling stable_deref_trait v1.1.1
   Compiling typenum v1.11.2
   Compiling smallvec v0.6.10
   Compiling num-traits v0.2.6
   Compiling scopeguard v0.3.3
   Compiling fnv v1.0.6
   Compiling crossbeam-utils v0.5.0
   Compiling num-integer v0.1.39
   Compiling fs_extra v1.1.0
   Compiling proc-macro2 v0.4.20
   Compiling crunchy v0.1.6
   Compiling parity-bytes v0.1.0
   Compiling opaque-debug v0.2.2
   Compiling byte-tools v0.3.1
   Compiling hash-db v0.15.2
   Compiling unicode-xid v0.1.0
   Compiling fake-simd v0.1.2
   Compiling subtle v1.0.0
   Compiling safemem v0.3.0
   Compiling heapsize v0.4.2
   Compiling arrayvec v0.4.11
   Compiling tiny-keccak v2.0.1
   Compiling nodrop v0.1.12
   Compiling subtle v2.2.2
   Compiling zeroize v1.1.0
   Compiling memchr v2.2.1
   Compiling matches v0.1.8
   Compiling unicode-normalization v0.1.7
   Compiling unexpected v0.1.0 (/project/util/unexpected)
   Compiling untrusted v0.7.0
   Compiling httparse v1.3.3
   Compiling linked-hash-map v0.5.1
   Compiling try-lock v0.2.2
   Compiling ansi_term v0.11.0
   Compiling percent-encoding v2.1.0
   Compiling openssl-probe v0.1.2
   Compiling bit-vec v0.4.4
   Compiling error-chain v0.12.0
   Compiling hamming v0.1.3
   Compiling unicode-segmentation v1.2.1
   Compiling nan-preserving-float v0.1.0
   Compiling memory_units v0.3.0
   Compiling primal-estimate v0.2.1
   Compiling ethabi-contract v9.0.0
   Compiling num-iter v0.1.37
   Compiling rustc-serialize v0.3.24
   Compiling siphasher v0.3.0
   Compiling arrayref v0.3.5
   Compiling version_check v0.9.1
   Compiling regex-syntax v0.6.12
   Compiling failure_derive v0.1.3
   Compiling rustc-demangle v0.1.9
   Compiling bitflags v1.0.4
   Compiling time-utils v0.1.0 (/project/util/time-utils)
   Compiling percent-encoding v1.0.1
   Compiling maplit v1.0.2
   Compiling glob v0.3.0
   Compiling version_check v0.1.5
   Compiling build_const v0.2.1
   Compiling ipnetwork v0.12.8
   Compiling using_queue v0.1.0 (/project/miner/using-queue)
   Compiling quick-error v1.2.2
   Compiling unicode-width v0.1.5
   Compiling vec_map v0.8.1
   Compiling strsim v0.8.0
   Compiling termcolor v1.0.4
   Compiling slab v0.2.0
   Compiling parity-path v0.1.1
   Compiling bindgen v0.49.2
   Compiling winapi v0.3.8
   Compiling utf8-ranges v1.0.4
   Compiling xdg v2.1.0
   Compiling peeking_take_while v0.1.2
   Compiling shlex v0.1.1
   Compiling mio-named-pipes v0.1.6
   Compiling edit-distance v2.0.1
   Compiling home v0.3.3
   Compiling lazycell v1.2.0
   Compiling remove_dir_all v0.5.1
   Compiling if_chain v0.1.3
   Compiling target_info v0.1.0
   Compiling slab v0.3.0
   Compiling winapi-build v0.1.1
   Compiling order-stat v0.1.3
   Compiling transient-hashmap v0.4.1
   Compiling winapi v0.2.8
   Compiling rprompt v1.0.3
   Compiling interleaved-ordered v0.1.1
   Compiling strsim v0.7.0
   Compiling thread_local v0.3.6
   Compiling crossbeam-utils v0.6.6
   Compiling crossbeam-utils v0.7.2
   Compiling crossbeam-epoch v0.8.2
   Compiling lock_api v0.3.1
   Compiling rlp v0.4.4
   Compiling c2-chacha v0.2.2
   Compiling rand_core v0.3.1
   Compiling rand_jitter v0.1.4
   Compiling hashbrown v0.6.3
   Compiling rand_pcg v0.1.2
   Compiling rand_chacha v0.1.1
   Compiling rand v0.6.5
   Compiling indexmap v1.3.0
   Compiling bincode v1.1.4
   Compiling owning_ref v0.3.3
   Compiling owning_ref v0.4.0
   Compiling tokio-service v0.1.0
   Compiling tokio-sync v0.1.6
   Compiling block-padding v0.1.4
   Compiling unicode-bidi v0.3.4
   Compiling itertools v0.8.2
   Compiling lru-cache v0.1.2
   Compiling bit-set v0.4.0
   Compiling primal-bit v0.2.4
   Compiling heck v0.3.0
   Compiling eip-152 v0.1.0 (/project/util/EIP-152)
   Compiling ethcore-bloom-journal v0.1.0 (/project/util/bloom)
   Compiling unicase v2.6.0
   Compiling xml-rs v0.7.0
   Compiling nom v4.2.3
   Compiling clang-sys v0.28.1
   Compiling humantime v1.1.1
   Compiling textwrap v0.11.0
   Compiling crc v1.8.1
   Compiling app_dirs v1.2.1 (https://github.com/paritytech/app-dirs-rs#0b37f948)
   Compiling tokio-timer v0.1.2
   Compiling textwrap v0.9.0
   Compiling kernel32-sys v0.2.2
   Compiling tokio-executor v0.1.8
   Compiling crossbeam-queue v0.1.2
   Compiling rand_core v0.2.2
   Compiling rand_isaac v0.1.1
   Compiling rand_xorshift v0.1.1
   Compiling rand_hc v0.1.0
   Compiling impl-rlp v0.2.1
   Compiling triehash v0.8.2
   Compiling lock_api v0.1.4
   Compiling idna v0.2.0
   Compiling idna v0.1.5
   Compiling primal-sieve v0.2.9
   Compiling xmltree v0.8.0
   Compiling tokio-current-thread v0.1.6
   Compiling tokio-timer v0.2.11
   Compiling num_cpus v1.10.1
   Compiling time v0.1.40
   Compiling atty v0.2.11
   Compiling quote v1.0.2
   Compiling iovec v0.1.2
   Compiling parking_lot_core v0.7.0
   Compiling net2 v0.2.33
   Compiling rand_os v0.1.3
   Compiling rand v0.4.6
   Compiling memmap v0.6.2
   Compiling socket2 v0.3.8
   Compiling fs-swap v0.2.4
   Compiling rpassword v1.0.2
   Compiling term_size v0.3.1
   Compiling fdlimit v0.1.1
   Compiling base64 v0.9.3
   Compiling base64 v0.10.1
   Compiling parity-wasm v0.31.3
   Compiling fxhash v0.2.1
   Compiling tiny-keccak v1.5.0
   Compiling uint v0.8.2
   Compiling want v0.0.6
   Compiling trace-time v0.1.1
   Compiling stats v0.1.0 (/project/util/stats)
   Compiling rand v0.5.5
   Compiling plain_hasher v0.2.0
   Compiling elastic-array v0.10.2
   Compiling num-traits v0.1.43
   Compiling number_prefix v0.2.8
   Compiling quote v0.6.8
   Compiling url v2.1.0
   Compiling aho-corasick v0.7.6
   Compiling aho-corasick v0.6.8
   Compiling url v1.7.1
   Compiling generic-array v0.12.0
   Compiling clap v2.33.0
   Compiling rand_core v0.5.1
   Compiling bytes v0.4.12
   Compiling parking_lot v0.10.0
   Compiling futures-cpupool v0.1.8
   Compiling mio v0.6.19
   Compiling miow v0.3.3
   Compiling tempdir v0.3.7
   Compiling toolshed v0.6.3
   Compiling transaction-pool v2.0.1
   Compiling wasmi v0.3.0
   Compiling pwasm-utils v0.6.1
   Compiling crossbeam-queue v0.2.1
   Compiling bn v0.4.4 (https://github.com/paritytech/bn#6beba2ed)
   Compiling rlp_compress v0.1.0 (/project/util/rlp-compress)
   Compiling enum_primitive v0.1.1
   Compiling chrono v0.4.6
   Compiling primal-check v0.2.3
   Compiling num-bigint v0.1.44
   Compiling syn v0.15.26
   Compiling regex v1.3.1
   Compiling mime v0.3.12
   Compiling mime_guess v2.0.1
   Compiling cexpr v0.3.6
   Compiling ctrlc v1.1.1 (https://github.com/paritytech/rust-ctrlc.git#b5230171)
   Compiling digest v0.8.0
   Compiling block-buffer v0.7.3
   Compiling block-cipher-trait v0.6.2
   Compiling crypto-mac v0.7.0
   Compiling stream-cipher v0.3.0
   Compiling rand_chacha v0.2.1
   Compiling rand_xorshift v0.2.0
   Compiling tokio-io v0.1.9
   Compiling http v0.1.21
   Compiling string v0.2.1
   Compiling len-caching-lock v0.1.1 (/project/util/len-caching-lock)
   Compiling mio-uds v0.6.7
   Compiling mio-extras v2.0.5
   Compiling synstructure v0.12.3
   Compiling timer v0.2.0
   Compiling primal v0.2.3
   Compiling vergen v3.0.4
   Compiling num v0.1.42
   Compiling globset v0.4.2
   Compiling env_logger v0.5.13
   Compiling synstructure v0.10.1
   Compiling env_logger v0.6.2
   Compiling sha2 v0.8.0
   Compiling ripemd160 v0.8.0
   Compiling sha-1 v0.8.1
   Compiling aes-soft v0.3.3
   Compiling block-modes v0.3.3
   Compiling hmac v0.7.0
   Compiling ctr v0.3.2
   Compiling rand v0.7.2
   Compiling tokio-codec v0.1.1
   Compiling h2 v0.1.26
   Compiling attohttpc v0.4.1
   Compiling serde_derive v1.0.103
   Compiling proc-macro-hack v0.5.9
   Compiling impl-trait-for-tuples v0.1.3
   Compiling derive_more v0.99.2
   Compiling rlp-derive v0.1.0
   Compiling ethcore-logger v1.12.0 (/project/parity/logger)
   Compiling aes v0.3.2
   Compiling pbkdf2 v0.3.0
   Compiling aes-ctr v0.3.0
   Compiling fixed-hash v0.5.2
   Compiling parity-wordlist v1.3.1
   Compiling ws v0.9.1
   Compiling igd v0.10.0
   Compiling parity-util-mem-derive v0.1.0
   Compiling logos-derive v0.7.7
   Compiling const-random-macro v0.1.6
   Compiling scrypt v0.2.0
   Compiling const-random v0.1.6
   Compiling ahash v0.2.18
   Compiling lru v0.4.3
   Compiling trie-db v0.20.0
   Compiling logos v0.7.7
   Compiling lunarity-lexer v0.2.1
   Compiling impl-serde v0.2.3
   Compiling parity-scale-codec v1.1.0
   Compiling serde_json v1.0.40
   Compiling semver v0.9.0
   Compiling docopt v1.0.2
   Compiling toml v0.5.6
   Compiling ethbloom v0.8.1
   Compiling jsonrpc-core v14.0.5
   Compiling validator v0.8.0
   Compiling rustc_version v0.2.3
   Compiling impl-codec v0.4.2
   Compiling proc-macro-crate v0.1.4
   Compiling blooms-db v0.1.0 (/project/util/blooms-db)
   Compiling jsonrpc-pubsub v14.0.6
   Compiling primitive-types v0.6.1
   Compiling memoffset v0.5.1
   Compiling parking_lot_core v0.3.1
   Compiling ethereum-types v0.8.0
   Compiling keccak-hash v0.4.0
   Compiling validator_derive v0.8.0
   Compiling keccak-hasher v0.1.1 (/project/util/keccak-hasher)
   Compiling ethjson v0.1.0 (/project/json)
   Compiling fastmap v0.1.0 (/project/util/fastmap)
   Compiling ethabi v9.0.1
   Compiling jsonrpc-derive v14.0.5
   Compiling patricia-trie-ethereum v0.1.0 (/project/util/patricia-trie-ethereum)
   Compiling triehash-ethereum v0.2.0 (/project/util/triehash-ethereum)
   Compiling parking_lot v0.6.4
   Compiling crossbeam-epoch v0.7.2
   Compiling vm v0.1.0 (/project/ethcore/vm)
   Compiling tokio-reactor v0.1.6
   Compiling failsafe v0.3.0
   Compiling crossbeam-deque v0.7.3
   Compiling crossbeam-deque v0.6.3
   Compiling wasm v0.1.0 (/project/ethcore/wasm)
   Compiling tokio-udp v0.1.2
   Compiling tokio-tcp v0.1.2
   Compiling tokio-uds v0.2.5
   Compiling tokio-threadpool v0.1.16
   Compiling rayon v1.3.0
   Compiling ethabi-derive v9.0.1
   Compiling tokio-fs v0.1.6
   Compiling cc v1.0.41
   Compiling tokio v0.1.22
   Compiling cmake v0.1.42
   Compiling ethcore-io v1.12.0 (/project/util/io)
   Compiling hyper v0.12.19
   Compiling parity-runtime v0.1.1
   Compiling jsonrpc-server-utils v14.0.3
   Compiling tokio-named-pipes v0.1.0
   Compiling jsonrpc-tcp-server v14.0.6
   Compiling jsonrpc-ws-server v14.0.6
   Compiling jemalloc-sys v0.3.2
   Compiling secp256k1-sys v0.1.2
   Compiling parity-snappy-sys v0.1.2
   Compiling ring v0.16.9
   Compiling backtrace-sys v0.1.32
   Compiling libloading v0.5.0
   Compiling natpmp v0.2.0
   Compiling parity-tokio-ipc v0.4.0
   Compiling jsonrpc-http-server v14.0.6
   Compiling ethcore-stratum v1.12.0 (/project/miner/stratum)
   Compiling jsonrpc-ipc-server v14.0.7
error: failed to run custom build command for `ring v0.16.9`

Caused by:
  process didn't exit successfully: `/target/release/build/ring-6569af80dd5b2ce0/build-script-build` (exit code: 101)
--- stdout
CARGO: /rust/bin/cargo
CARGO_CFG_TARGET_ARCH: mips
CARGO_CFG_TARGET_ENDIAN: big
CARGO_CFG_TARGET_ENV: gnu
CARGO_CFG_TARGET_FAMILY: unix
CARGO_CFG_TARGET_HAS_ATOMIC: 16,32,8,ptr
CARGO_CFG_TARGET_HAS_ATOMIC_LOAD_STORE: 16,32,8,ptr
CARGO_CFG_TARGET_OS: linux
CARGO_CFG_TARGET_POINTER_WIDTH: 32
CARGO_CFG_TARGET_THREAD_LOCAL: 
CARGO_CFG_TARGET_VENDOR: unknown
CARGO_CFG_UNIX: 
CARGO_FEATURE_ALLOC: 1
CARGO_FEATURE_DEFAULT: 1
CARGO_FEATURE_DEV_URANDOM_FALLBACK: 1
CARGO_FEATURE_LAZY_STATIC: 1
CARGO_HOME: /cargo
CARGO_MAKEFLAGS: --jobserver-fds=19,22 -j --jobserver-auth=19,22 -j
CARGO_MANIFEST_DIR: /cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.9
CARGO_MANIFEST_LINKS: ring-asm
CARGO_PKG_AUTHORS: Brian Smith <brian@briansmith.org>
CARGO_PKG_DESCRIPTION: Safe, fast, small crypto using Rust.
CARGO_PKG_HOMEPAGE: 
CARGO_PKG_NAME: ring
CARGO_PKG_REPOSITORY: https://github.com/briansmith/ring
CARGO_PKG_VERSION: 0.16.9
CARGO_PKG_VERSION_MAJOR: 0
CARGO_PKG_VERSION_MINOR: 16
CARGO_PKG_VERSION_PATCH: 9
CARGO_PKG_VERSION_PRE: 
CARGO_TARGET_DIR: /target
CARGO_TARGET_MIPS_UNKNOWN_LINUX_GNU_LINKER: mips-linux-gnu-gcc
CARGO_TARGET_MIPS_UNKNOWN_LINUX_GNU_RUNNER: /linux-runner mips
CC_mips_unknown_linux_gnu: mips-linux-gnu-gcc
CROSS_RUNNER: 
CXX_mips_unknown_linux_gnu: mips-linux-gnu-g++
DEBUG: false
HOME: /
HOST: x86_64-unknown-linux-gnu
HOSTNAME: 798e4444dd89
LD_LIBRARY_PATH: /target/release/deps:/rust/lib
NUM_JOBS: 6
OPT_LEVEL: 3
OUT_DIR: /target/mips-unknown-linux-gnu/release/build/ring-776b5fd8ead9bde1/out
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/rust/bin
PROFILE: release
PWD: /project
QEMU_LD_PREFIX: /usr/mips-linux-gnu
QEMU_STRACE: 1
RUSTC: rustc
RUSTC_LINKER: mips-linux-gnu-gcc
RUSTDOC: rustdoc
RUST_BACKTRACE: 1
RUST_TEST_THREADS: 1
SSL_CERT_DIR: /usr/lib/ssl/certs
SSL_CERT_FILE: /usr/lib/ssl/certs/ca-certificates.crt
TARGET: mips-unknown-linux-gnu
TERM: xterm
USER: ls
XARGO_HOME: /xargo

--- stderr
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.9/build.rs:364:34
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/libunwind.rs:86
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:78
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:59
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1053
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1428
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:62
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:204
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:224
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:470
  11: rust_begin_unwind
             at src/libstd/panicking.rs:378
  12: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
  13: core::panicking::panic
             at src/libcore/panicking.rs:52
  14: build_script_build::main
  15: std::rt::lang_start::{{closure}}
  16: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:52
  17: std::panicking::try::do_call
             at src/libstd/panicking.rs:303
  18: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:86
  19: std::panicking::try
             at src/libstd/panicking.rs:281
  20: std::panic::catch_unwind
             at src/libstd/panic.rs:394
  21: std::rt::lang_start_internal
             at src/libstd/rt.rs:51
  22: main
  23: __libc_start_main
  24: _start
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

warning: build failed, waiting for other jobs to finish...
error: build failed
dvdplm commented 4 years ago

thread 'main' panicked at 'called Option::unwrap() on a None value', /cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.9/build.rs:364:34

Did you try building ring in isolation?

ordian commented 4 years ago

open-ethereum currently doesn't support big-endian platforms https://github.com/OpenEthereum/open-ethereum/blob/3ccfe735aa579f89a1f8de192a7625084a6f530f/ethash/src/compute.rs#L265-L268

we also might have problems with parity-common crates https://github.com/paritytech/parity-common/issues/27

ltfschoen commented 4 years ago

I was reading through the "ring" README file, and they mention

Sometimes things are broken with Nightly Rust. We prioritize keeping things working on Stable So I tried recompiling on Rust "stable" but encountered the same error.

They also say in BUILD.md of the "ring" repo:

ring targets the current stable release of Rust and Cargo. We also verify that the current beta and nightly releases work

I'm not sure what's using "ring" as a dependency, but it's using version ring-0.16.9 https://github.com/briansmith/ring/commit/56cddd73d20fc440007c893cbd49d9e7633ae697, whereas the latest version of ring is 0.16.11 https://github.com/briansmith/ring/commit/da4f7ba2b955fbf8e37fe652ab6c26527564f52f, so perhaps 0.16.9 doesn't support the latest version of Rust stable or nightly version of Rust that I'm using, which is rustc 1.41.1 (f3e1a954d 2020-02-24).

ltfschoen commented 4 years ago

@dvdplm I tried building ring-0.16.11 from source and it builds successfully with the latest Rust stable version rustc 1.41.1 (f3e1a954d 2020-02-24)

But when I try to build the older ring-0.16.9 from source when using the latest Rust stable version rustc 1.41.1 (f3e1a954d 2020-02-24), it fails with the following error.

ls @ ~/code/src/ltfschoen/ring - [56cddd73d20fc440007c893cbd49d9e7633ae697] $ cargo build --release
   Compiling ring v0.16.9 (/Users/ls/code/src/ltfschoen/ring)
warning: lint `legacy_directory_ownership` has been removed: `converted into hard error, see https://github.com/rust-lang/rust/issues/37872`
  --> build.rs:20:5
   |
20 |     legacy_directory_ownership,
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(renamed_and_removed_lints)]` on by default

error: this method call currently resolves to `<&[T; N] as IntoIterator>::into_iter` (due to autoref coercions), but that might change in the future when `IntoIterator` impls for arrays are added.
   --> build.rs:421:10
    |
421 |     libs.into_iter()
    |          ^^^^^^^^^ help: use `.iter()` instead of `.into_iter()` to avoid ambiguity: `iter`
    |
note: lint level defined here
   --> build.rs:33:5
    |
33  |     warnings
    |     ^^^^^^^^
    = note: `#[forbid(array_into_iter)]` implied by `#[forbid(warnings)]`
    = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
    = note: for more information, see issue #66145 <https://github.com/rust-lang/rust/issues/66145>

error: aborting due to previous error

error: could not compile `ring`.

To learn more, run the command again with --verbose.

So I had a look at Open Ethereum's cargo.lock file, and the following dependencies are using "ring":

[[package]]
name = "webpki"
version = "0.21.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7e664e770ac0110e2384769bcc59ed19e329d81f555916a6e072714957b81b4"
dependencies = [
 "ring",
 "untrusted",
]

[[package]]
name = "rustls"
version = "0.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b25a18b1bf7387f0145e7f8324e700805aade3842dd3db2e74e4cdeb4677c09e"
dependencies = [
 "base64 0.10.1",
 "log",
 "ring",
 "sct",
 "webpki",
]

[[package]]
name = "sct"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3042af939fca8c3453b7af0f1c66e533a15a86169e39de2657310ade8f98d3c"
dependencies = [
 "ring",
 "untrusted",
]

And the following dependency uses "webpki" and "rustls" (where "rustls" uses "sct"):

[[package]]
name = "hyper-rustls"
version = "0.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b66a4973381d01141ed0a4f20070d47a232c764642dd6217d3d93f7a1f952ea5"
dependencies = [
 "bytes",
 "ct-logs",
 "futures",
 "hyper",
 "rustls",
 "rustls-native-certs",
 "tokio-io",
 "tokio-rustls",
 "webpki",
]

And "hyper-rustls" is in a Cargo.toml file, so perhaps if we update that to its latest stable version then the other dependencies (i.e. "webpki", "rustls" (which replaced "openssl" library, see here), and "sct") will also use their latest versions, and then it's just a matter of making updates to Open Ethereum codebase to make it build with the latest versions. And I can't find any Open Ethereum Issues that mention there being a requirement to update "hyper-rustls".

Should a new issue be added to update Open Ethereum to use the latest stable version of "hyper-rustls"?

Then it occurred to me that perhaps doing that already an existing Open Ethereum Issue...

So first I found the following Issue that appears to indicate the repo is due for an update to the "ring" dependency https://github.com/OpenEthereum/open-ethereum/issues/11544

And then I found this issue https://github.com/OpenEthereum/open-ethereum/issues/7915. @tomaka are you still going to use "ring" to resolve this issue even though it's a "problematic library"?

And I also found this issue which appears to have been caused by "rustls-native-certs", which is a dependency of "hyper-rustls" https://github.com/OpenEthereum/open-ethereum/issues/11515

ltfschoen commented 4 years ago

I've since tried reverting to an older version ring-0.16.5 (with rustc 1.36.0) unsuccessfully. I've also tried updating to hyper-rustls 0.20.0 unsuccessfully.

niklasad1 commented 4 years ago
  1. Ring doesn't support MIPS currently but there is ongoing work
  2. The reason why ring is problematic is that the majority of implementations depend on assembly rather than source, thus not cross-platform and doesn't work for WASM, MIPS or similar.
  3. We don't depend on OpenSSL anymore more than for certificates from hyper-rustls
  4. The errors you received are caused by a lint triggered by a newer toolchain, master should work but you will receive another error :)

And I also found this issue which appears to have been caused by "rustls-native-certs", which is a dependency of "hyper-rustls" #11515

Great, thanks

It would be great if you could create another tracking issue/rename this one which summaries what need to be done in order to get OpenEthereum to compile for MIPS/BE :)

ltfschoen commented 4 years ago

@niklasad1 I've created a tracking issue here https://github.com/openethereum/openethereum/issues/11563

knarz commented 4 years ago

Hey, this branch https://github.com/knarz/ring/tree/bigendian has a patch that makes ring work on the s360 arch and is applied to the 0.16.9 version of the crate.

Edit: Not s360, but s390x, sorry

ltfschoen commented 4 years ago

@knarz i tried forking https://github.com/knarz/ring and then cloning the fork and fetching the 'bigendian' branch git fetch origin bigendian:bigendian && git checkout bigendian. I checked if s360 arch was supported by cross, but the closest I could find here was s390x. So I ran RUST_BACKTRACE=1 QEMU_STRACE=1 cross build --target s390x-unknown-linux-gnu --release, which required a small change https://github.com/knarz/ring/compare/bigendian...ltfschoen:bigendian?expand=1, and then when I ran it again I got the following error: 

ls @ ~/code/src/ltfschoen/ring - [bigendian] $ RUST_BACKTRACE=1 QEMU_STRACE=1 cross build --target s390x-unknown-linux-gnu --release
info: downloading component 'rust-std' for 's390x-unknown-linux-gnu'
 19.5 MiB /  19.5 MiB (100 %)   4.1 MiB/s in  5s ETA:  0s
info: installing component 'rust-std' for 's390x-unknown-linux-gnu'
Unable to find image 'rustembedded/cross:s390x-unknown-linux-gnu-0.2.0' locally
s390x-unknown-linux-gnu-0.2.0: Pulling from rustembedded/cross
fe703b657a32: Already exists 
f9df1fafd224: Already exists 
a645a4b887f9: Already exists 
57db7fe0b522: Already exists 
fd5eb1dae638: Pull complete 
b737b1aa458a: Pull complete 
ebe922695700: Pull complete 
25f818714196: Pull complete 
51846252d77b: Pull complete 
33501c307ac6: Pull complete 
45f447272a84: Pull complete 
b4cfcdce639c: Pull complete 
cc1ce785c1b0: Pull complete 
d4b535e6cdc5: Pull complete 
76878529c2cc: Pull complete 
b8fe6886ab5e: Pull complete 
a7e971cfb355: Pull complete 
3261ee26d2d9: Pull complete 
Digest: sha256:8840442382fa84b947087cec646f81802c28a425c365ce3c32b51bf5bf924e19
Status: Downloaded newer image for rustembedded/cross:s390x-unknown-linux-gnu-0.2.0
   Compiling untrusted v0.7.0
   Compiling lazy_static v1.4.0
   Compiling libc v0.2.68
warning: lint `legacy_directory_ownership` has been removed: `converted into hard error, see https://github.com/rust-lang/rust/issues/37872`
  --> build.rs:20:5
   |
20 |     legacy_directory_ownership,
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(renamed_and_removed_lints)]` on by default

   Compiling ring v0.16.9 (/project)
error: failed to run custom build command for `ring v0.16.9 (/project)`

Caused by:
  process didn't exit successfully: `/target/release/build/ring-f66cff03479ab192/build-script-build` (exit code: 101)
--- stdout
CARGO: /rust/bin/cargo
CARGO_CFG_DEBUG_ASSERTIONS: 
CARGO_CFG_PROC_MACRO: 
CARGO_CFG_TARGET_ARCH: s390x
CARGO_CFG_TARGET_ENDIAN: big
CARGO_CFG_TARGET_ENV: gnu
CARGO_CFG_TARGET_FAMILY: unix
CARGO_CFG_TARGET_OS: linux
CARGO_CFG_TARGET_POINTER_WIDTH: 64
CARGO_CFG_TARGET_VENDOR: unknown
CARGO_CFG_UNIX: 
CARGO_FEATURE_ALLOC: 1
CARGO_FEATURE_DEFAULT: 1
CARGO_FEATURE_DEV_URANDOM_FALLBACK: 1
CARGO_FEATURE_LAZY_STATIC: 1
CARGO_HOME: /cargo
CARGO_MAKEFLAGS: --jobserver-fds=5,6 -j --jobserver-auth=5,6 -j
CARGO_MANIFEST_DIR: /project
CARGO_MANIFEST_LINKS: ring-asm
CARGO_PKG_AUTHORS: Brian Smith 
CARGO_PKG_DESCRIPTION: Safe, fast, small crypto using Rust.
CARGO_PKG_HOMEPAGE: 
CARGO_PKG_NAME: ring
CARGO_PKG_REPOSITORY: https://github.com/briansmith/ring
CARGO_PKG_VERSION: 0.16.9
CARGO_PKG_VERSION_MAJOR: 0
CARGO_PKG_VERSION_MINOR: 16
CARGO_PKG_VERSION_PATCH: 9
CARGO_PKG_VERSION_PRE: 
CARGO_TARGET_DIR: /target
CARGO_TARGET_S390X_UNKNOWN_LINUX_GNU_LINKER: s390x-linux-gnu-gcc
CARGO_TARGET_S390X_UNKNOWN_LINUX_GNU_RUNNER: /linux-runner s390x
CC_s390x_unknown_linux_gnu: s390x-linux-gnu-gcc
CROSS_RUNNER: 
CXX_s390x_unknown_linux_gnu: s390x-linux-gnu-g++
DEBUG: false
HOME: /
HOST: x86_64-unknown-linux-gnu
HOSTNAME: 70399a773b57
LD_LIBRARY_PATH: /target/release/deps:/rust/lib
NUM_JOBS: 6
OPT_LEVEL: 3
OUT_DIR: /target/s390x-unknown-linux-gnu/release/build/ring-0f5362b98cee1d36/out
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/rust/bin
PROFILE: release
PWD: /project
QEMU_LD_PREFIX: /usr/s390x-linux-gnu
QEMU_STRACE: 1
RUSTC: rustc
RUSTC_LINKER: s390x-linux-gnu-gcc
RUSTDOC: rustdoc
RUST_TEST_THREADS: 1
SSL_CERT_DIR: /usr/lib/ssl/certs
SSL_CERT_FILE: /usr/lib/ssl/certs/ca-certificates.crt
TARGET: s390x-unknown-linux-gnu
TERM: xterm
USER: ls
XARGO_HOME: /xargo

--- stderr
thread 'main' panicked at '"include/GFp/s390x_arch.h": Os { code: 2, kind: NotFound, message: "No such file or directory" }', build.rs:771:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

warning: build failed, waiting for other jobs to finish...
error: build failed

I also forked 'sct.rs', 'webpki', and 'rustls'. The plan was to update Cargo.toml in each of them as follows:

Then lastly to update 'open-ethereum' (my fork) dependencies to use my forks, as follows, when they are ready:

hyper-rustls = { git = "https://github.com/ltfschoen/ring", branch = "bigendian" }
ltfschoen commented 4 years ago

as an alternative to the above, i'm in the process of getting access to a IBM Z mainframe via VM by registering for community access at https://linuxone.cloud.marist.edu/cloud/#/register?flag=vm (instead of running on macOS)

knarz commented 4 years ago

Hey, I made a mistake. Of course it's s390x and not s360. In the Cargo.toml I added the following:

[patch.crates-io]
ring = { path = "../ring"}

That should replace all ring crates with the patched version, so no need to fork everything.

knarz commented 4 years ago

I published some forks, that allow the client to be built on the s390x using a couple of patched crates.

https://github.com/knarz/parity-common/tree/bigendian https://github.com/knarz/openethereum/tree/ring

The above openethereum fork will expect ring and parity-common directories to be available in ../. (See the Cargo.toml in the openethereum client).

With these patches, starting the client once works but shutting it down and restarting again fails. This is just the output of cargo run -- -l "debug" after restarting once:

2020-03-24 03:24:10  tokio-runtime-worker-0 DEBUG tokio_reactor  loop process - 0 events, 0.000s
2020-03-24 03:24:10  tokio-runtime-worker-0 DEBUG tokio_reactor  loop process - 1 events, 0.000s
2020-03-24 03:24:10  tokio-runtime-worker-0 DEBUG tokio_reactor  loop process - 0 events, 0.000s
2020-03-24 03:24:10  fetch DEBUG fetch  processing requests ...
2020-03-24 03:24:11  main WARN kvdb_rocksdb  DB corrupted: Invalid argument: You have to open all column families. Column families not opened: col8, col7, col6, col1, col0, col2, col3, col4, col5, attempting repair
2020-03-24 03:24:11  fetch DEBUG tokio_reactor  loop process - 1 events, 0.000s
2020-03-24 03:24:11  fetch DEBUG fetch  fetch background thread finished
2020-03-24 03:24:11  tokio-runtime-worker-0 DEBUG tokio_reactor  loop process - 1 events, 0.000s
2020-03-24 03:24:11  tokio-runtime-worker-0 DEBUG tokio_reactor  loop process - 0 events, 0.000s
2020-03-24 03:24:11  tokio-runtime-worker-0 DEBUG tokio_reactor  loop process - 1 events, 0.000s
Failed to open database Custom { kind: Other, error: Error { message: "Invalid argument: Column family not found: : col5" } }

With a PoA demo setup (as per https://wiki.parity.io/Demo-PoA-tutorial.html) I get a strange error I haven't been able to track down yet:

2020-03-24 03:29:02  main INFO ethcore_service::service  Configured for DemoPoA using AuthorityRound engine

====================

stack backtrace:
   0: panic_hook::gen_panic_msg
             at util/panic-hook/src/lib.rs:71
   1: panic_hook::set_with::{{closure}}
             at util/panic-hook/src/lib.rs:44
   2: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:476
   3: rust_begin_unwind
             at src/libstd/panicking.rs:380
   4: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
   5: core::option::expect_none_failed
             at src/libcore/option.rs:1199
   6: core::result::Result<T,E>::expect
             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libcore/result.rs:991
   7: journaldb::overlayrecentdb::OverlayRecentDB::read_overlay
             at util/journaldb/src/overlayrecentdb.rs:180
   8: journaldb::overlayrecentdb::OverlayRecentDB::new
             at util/journaldb/src/overlayrecentdb.rs:147
   9: journaldb::new
             at util/journaldb/src/lib.rs:195
  10: ethcore::client::client::Client::new
             at ethcore/src/client/client.rs:723
  11: ethcore_service::service::ClientService::start
             at ethcore/service/src/service.rs:123
  12: open_ethereum::run::execute_impl
             at /home/pocadm2/openethereum/parity/run.rs:563
  13: open_ethereum::run::execute
             at /home/pocadm2/openethereum/parity/run.rs:930
  14: open_ethereum::execute
             at /home/pocadm2/openethereum/parity/lib.rs:205
  15: open_ethereum::start
             at /home/pocadm2/openethereum/parity/lib.rs:249
  16: parity::main_direct
             at parity/main.rs:276
  17: parity::main
             at parity/main.rs:453
  18: std::rt::lang_start::{{closure}}
             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libstd/rt.rs:67
  19: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:52
      std::panicking::try::do_call
             at src/libstd/panicking.rs:305
  20: __rust_try
  21: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:86
  22: std::panicking::try
             at src/libstd/panicking.rs:281
      std::panic::catch_unwind
             at src/libstd/panic.rs:394
      std::rt::lang_start_internal
             at src/libstd/rt.rs:51
  23: std::rt::lang_start
             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libstd/rt.rs:67
  24: main
  25: __libc_start_main
  26: <unknown>

Thread 'main' panicked at 'Low-level database error.: Custom { kind: Other, error: Error { message: "Corruption: Snappy not supported or corrupted Snappy compressed block contents" } }', util/journaldb/src/overlayrecentdb.rs:180

This is a bug. Please report it at:

    https://github.com/OpenEthereum/open-ethereum/issues/new

Aborted