Error: Failed to extract data from archive member

[jhugman]

I'm trying to use library_mode to generate_bindings from a reasonably large crate, the matrix-sdk-ffi bindings.

Also relevant: I've bumped uniffi to version 0.28.2.

Using cargo ndk, I'm building the staticlib version for aarc64-linux-android, in release mode only.

  cargo ndk \
  --manifest-path ./bindings/matrix-sdk-ffi/Cargo.toml \
  --target arm64-v8a \
  --platform 21 --no-strip -- \
  build --release

Then using print-repr to look at the metadata.

cargo run \
  --manifest-path uniffi-bindgen/Cargo.toml \
  print-repr \
  target/aarch64-linux-android/release/libmatrix_sdk_ffi.a 

Output is:

    Finished `dev` profile [unoptimized] target(s) in 0.13s
     Running `target/debug/uniffi-bindgen print-repr target/aarch64-linux-android/release/libmatrix_sdk_ffi.a`
Failed to extract data from archive member `matrix_sdk_ffi.matrix_sdk_ffi.bf8c76dbb4572677-cgu.07.rcgu.o`

A lightly instrumented uniffi which prints in extract_item that a symbol has been recognized, but is an unrecognized metadata code:

Metadata symbol UNIFFI_META_MATRIX_SDK_FFI_INTERFACE_ENCRYPTION
thread 'main' panicked at uniffi_meta/src/metadata.rs:84:18:
XXXX UNKNOWN ITEM 88 XXXX
stack backtrace:
   0: rust_begin_unwind
             at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/panicking.rs:647:5
   1: core::panicking::panic_fmt
             at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/core/src/panicking.rs:72:14
   2: uniffi_meta::metadata::codes::metadata_as_str
             at ./uniffi_meta/src/metadata.rs:84:18
   3: uniffi_meta::reader::MetadataReader::read_metadata
             at ./uniffi_meta/src/reader.rs:39:33
   4: uniffi_meta::reader::read_metadata
             at ./uniffi_meta/src/reader.rs:10:5
   5: uniffi_meta::Metadata::read
             at ./uniffi_meta/src/lib.rs:505:9
   6: uniffi_bindgen::macro_metadata::extract::ExtractedItems::extract_item
             at ./uniffi_bindgen/src/macro_metadata/extract.rs:184:25
   7: uniffi_bindgen::macro_metadata::extract::extract_from_elf
             at ./uniffi_bindgen/src/macro_metadata/extract.rs:53:13
   8: uniffi_bindgen::macro_metadata::extract::extract_from_bytes
             at ./uniffi_bindgen/src/macro_metadata/extract.rs:29:29
   9: uniffi_bindgen::macro_metadata::extract::extract_from_archive
             at ./uniffi_bindgen/src/macro_metadata/extract.rs:143:18
  10: uniffi_bindgen::macro_metadata::extract::extract_from_bytes
             at ./uniffi_bindgen/src/macro_metadata/extract.rs:32:37
  11: uniffi_bindgen::macro_metadata::extract::extract_from_library
             at ./uniffi_bindgen/src/macro_metadata/extract.rs:24:5
  12: uniffi_bindgen::library_mode::find_components
             at ./uniffi_bindgen/src/library_mode.rs:104:17
  13: uniffi_bindgen::library_mode::generate_bindings
             at ./uniffi_bindgen/src/library_mode.rs:46:26
  14: uniffi::cli::uniffi_bindgen::gen_library_mode
             at ./uniffi/src/cli/uniffi_bindgen.rs:207:38
  15: uniffi::cli::uniffi_bindgen::run_main
             at ./uniffi/src/cli/uniffi_bindgen.rs:297:17
  16: uniffi::cli::uniffi_bindgen_main
             at ./uniffi/src/cli/mod.rs:9:21
  17: uniffi_bindgen::main
             at ./cli/src/main.rs:2:5
  18: core::ops::function::FnOnce::call_once
             at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Unfortunately, I don't understand ELF, ndk or the process of it being written to move the investigation further.

---

Investigation to get to here: https://github.com/jhugman/uniffi-bindgen-react-native/issues/122

[mhammond]

That sounds a little like a mismatch between the bytes written by that item vs the bytes we think we should read. Can you share the definition of UNIFFI_META_MATRIX_SDK_FFI_INTERFACE_ENCRYPTION as emitted by the macros?

[jhugman]

Using cargo expand, I'm getting:

    const UNIFFI_META_CONST_MATRIX_SDK_FFI_INTERFACE_ENCRYPTION: ::uniffi::MetadataBuffer = ::uniffi::MetadataBuffer::from_code(
            ::uniffi::metadata::codes::INTERFACE,
        )
        .concat_str("matrix_sdk_ffi")
        .concat_str("Encryption")
        .concat_long_str("");
    #[no_mangle]
    #[doc(hidden)]
    pub static UNIFFI_META_MATRIX_SDK_FFI_INTERFACE_ENCRYPTION: [u8; UNIFFI_META_CONST_MATRIX_SDK_FFI_INTERFACE_ENCRYPTION
        .size] = UNIFFI_META_CONST_MATRIX_SDK_FFI_INTERFACE_ENCRYPTION.into_array();

[mhammond]

huh, that seems correct. After we read ::uniffi::metadata::codes::INTERFACE here, we then read an object here, which tries to read 3 strings - which looks exactly like what we are emitting. So sorry, but that doesn't help me understand what's going wrong :(

[jhugman]

Yeah, the reading/writing of the metadata works just fine in debug, and with other architectures.

I would guess (and I am guessing here), is that there is some subtlety in the extract_elf method that works in debug OR lib.so, but not in release AND lib.a.

@bendk pointed out a fix for a similar looking problem in MachO builds.