feat/KotlinUnifi: building with `./build-kotlin.sh` fails

[dbluhm]

While attempting to build the Kotlin wrapper from the https://github.com/Indicio-tech/aries-askar/tree/feat/KotlinUniffi branch with ./build-kotlin.sh, the script fails while compiling (during cross build)

[KolbyRKunz]

The build-kotlin.sh file was initially used to create the kotlin bindings, those bindings are already included in the wrapper as it is right now and so you do not necessarily have to run that script. It is important to note that you need to have docker desktop installed and running in order for the cross command in the scripts to run.

The better alternative since the bindings are already generated is the build-targets.sh file inside of the ./wrapper/kotlin folder. This will build for iOS and Android and the resulting build files will be in the ./targets folder. The next step will be to run the build-universal.sh file in the root of the project. This builds a universal dylib for macOS that allows you to run the tests on your machine rather than a simulator or jvm. If for some reason you do not want to do that the build.gradle.kts will have to be adjusted to not try to copy from that folder and then comment the macOS section out.

Once you have all of the binaries built you must create a jniLibs folder inside of ./wrapper/kotlin/src/androidMain and create four folders and copy the correct binaries into the folder. The folders to create are "arm64-v8a", "armeabi-v7a", "x86", "x86_64" then from the corresponding folders in the ./targets you need to copy the libaries_askar.so and libaries_askar.a into each of the folders you created, just the .so and .a from the matching dir in ./target are required.

Once that is done you should be able to open the ./wrappers/kotlin in an IDE, we used intellij idea, and run the tests. If you used idea then you will want to make sure the IDE creates a local.properties file that has a variable pointing to your android sdk install location. "sdk.dir="Path/on/your/machine".

One thing to note is that with idea we are able to run the tests on either macOS, jvm, or iOS simulator. To run on android the tests are configured to run as android instrumented tests which require a physical device to be plugged into your machine. This is just done by changing the run configuration in the top right to run android instrumented all tests. Once all tests are running and working the project should be able to be used as a dependency for another project. There is some configuration in the build.gradle.kts of the project to publish the project to a maven repo but it has not yet been done and requires additional set up to publish.

[KolbyRKunz]

Building all of the targets can take over an hour.

[dkulic]

I was able to build all, but after a several retries. It fails randomly, but continues after restart. I tried on MacOS (M1).

[KolbyRKunz]

Strange, do you have any of the logs from the fails available?

[dkulic]

The issues are very strange indeed, here is the output, I restarted it from begining:

    Updating crates.io index
    Updating git repository `https://github.com/mozilla/uniffi-rs.git`
    Updating git repository `https://github.com/cross-rs/cross`
  Installing cross v0.2.5 (https://github.com/cross-rs/cross#96a8da5f)
    Updating crates.io index
  Downloaded eyre v0.6.9
  Downloaded serde_derive v1.0.193
  Downloaded serde v1.0.193
  Downloaded errno v0.3.7
  Downloaded rustix v0.38.25
  Downloaded 5 crates (549.2 KB) in 0.68s
   Compiling libc v0.2.150
   Compiling proc-macro2 v1.0.69
   Compiling unicode-ident v1.0.12
   Compiling serde v1.0.193
   Compiling rustix v0.38.25
   Compiling utf8parse v0.2.1
   Compiling cfg-if v1.0.0
   Compiling bitflags v2.4.1
   Compiling adler v1.0.2
   Compiling colorchoice v1.0.0
   Compiling anstyle-parse v0.2.2
   Compiling gimli v0.28.0
   Compiling equivalent v1.0.1
   Compiling hashbrown v0.14.2
   Compiling memchr v2.6.4
   Compiling semver v1.0.20
   Compiling anstyle v1.0.4
   Compiling is_ci v1.1.1
   Compiling anstyle-query v1.0.0
   Compiling eyre v0.6.9
   Compiling indexmap v2.1.0
   Compiling anstream v0.6.4
   Compiling object v0.32.1
   Compiling miniz_oxide v0.7.1
   Compiling indenter v0.3.3
   Compiling signal-hook v0.3.17
   Compiling addr2line v0.21.0
   Compiling winnow v0.5.19
   Compiling once_cell v1.18.0
   Compiling rustc-demangle v0.1.23
   Compiling heck v0.4.1
   Compiling thiserror v1.0.50
   Compiling serde_json v1.0.108
   Compiling clap_lex v0.6.0
   Compiling strsim v0.10.0
   Compiling clap_builder v4.4.8
   Compiling itoa v1.0.9
   Compiling bitflags v1.3.2
   Compiling either v1.9.0
   Compiling home v0.5.5
   Compiling fastrand v2.0.1
   Compiling ryu v1.0.15
   Compiling cross v0.2.5 (/Users/dkulic/.cargo/git/checkouts/cross-dac8861107f29545/96a8da5)
   Compiling quote v1.0.33
   Compiling syn v2.0.39
   Compiling rustc_version v0.4.0
   Compiling shell-escape v0.1.5
   Compiling errno v0.3.7
   Compiling atty v0.2.14
   Compiling cc v1.0.83
   Compiling supports-color v1.3.1
   Compiling signal-hook-registry v1.4.1
   Compiling owo-colors v3.5.0
   Compiling dirs-sys v0.3.7
   Compiling nix v0.26.4
   Compiling directories v4.0.1
   Compiling const-sha1 v0.2.0
   Compiling shell-words v1.1.0
   Compiling backtrace v0.3.69
   Compiling tempfile v3.8.1
   Compiling which v4.4.2
   Compiling is-terminal v0.4.9
   Compiling color-eyre v0.6.2
   Compiling serde_derive v1.0.193
   Compiling clap_derive v4.4.7
   Compiling thiserror-impl v1.0.50
   Compiling clap v4.4.8
   Compiling serde_spanned v0.6.4
   Compiling toml_datetime v0.6.5
   Compiling serde_ignored v0.1.9
   Compiling toml_edit v0.19.15
   Compiling toml v0.7.8
    Finished release [optimized] target(s) in 42.63s
   Replacing /Users/dkulic/.cargo/bin/cross
   Replacing /Users/dkulic/.cargo/bin/cross-util
    Replaced package `cross v0.2.5 (https://github.com/cross-rs/cross#44011c88)` with `cross v0.2.5 (https://github.com/cross-rs/cross#96a8da5f)` (executables `cross`, `cross-util`)
Building for aarch64-linux-android...
info: syncing channel updates for '1.65.0-aarch64-apple-darwin'
info: latest update on 2022-11-03, rust version 1.65.0 (897e37553 2022-11-02)
info: component 'rust-std' for target 'aarch64-linux-android' is up to date

  1.65.0-aarch64-apple-darwin unchanged - rustc 1.65.0 (897e37553 2022-11-02)

[cross] warning: using newer rustc `1.73.0 (cc66ad468 2023-10-03)` for the target. Current active rustc on the host is `rustc 1.72.1 (d5c2e9c34 2023-09-13)`.
 > Update with `rustup update`
   Compiling proc-macro2 v1.0.69
   Compiling unicode-ident v1.0.12
   Compiling libc v0.2.150
   Compiling version_check v0.9.4
   Compiling autocfg v1.1.0
   Compiling typenum v1.17.0
   Compiling cfg-if v1.0.0
   Compiling serde v1.0.193
   Compiling subtle v2.5.0
   Compiling futures-core v0.3.29
   Compiling thiserror v1.0.50
   Compiling serde_json v1.0.108
   Compiling paste v1.0.14
   Compiling generic-array v0.14.7
   Compiling lock_api v0.4.11
   Compiling memchr v2.6.4
   Compiling syn v1.0.109
   Compiling unicase v2.7.0
   Compiling getrandom v0.2.11
   Compiling minimal-lexical v0.2.1
   Compiling quote v1.0.33
   Compiling cc v1.0.83
   Compiling const-oid v0.9.5
   Compiling once_cell v1.18.0
   Compiling syn v2.0.39
   Compiling rand_core v0.6.4
   Compiling nom v7.1.3
   Compiling camino v1.1.6
   Compiling semver v1.0.20
   Compiling ring v0.17.5
   Compiling num-traits v0.2.17
   Compiling ahash v0.8.6
   Compiling anyhow v1.0.75
   Compiling mime_guess v2.0.4
   Compiling ryu v1.0.15
   Compiling pin-project-lite v0.2.13
   Compiling itoa v1.0.9
   Compiling slab v0.4.9
   Compiling bytes v1.5.0
   Compiling futures-task v0.3.29
   Compiling log v0.4.20
   Compiling parking_lot_core v0.9.9
   Compiling cpufeatures v0.2.11
   Compiling serde_derive v1.0.193
   Compiling zeroize_derive v1.4.2
   Compiling thiserror-impl v1.0.50
   Compiling scopeguard v1.2.0
   Compiling fs-err v2.11.0
   Compiling crossbeam-utils v0.8.16
   Compiling futures-util v0.3.29
   Compiling futures-io v0.3.29
   Compiling mime v0.3.17
   Compiling scroll_derive v0.11.1
   Compiling mio v0.8.9
   Compiling socket2 v0.5.5
   Compiling num_cpus v1.16.0
   Compiling futures-channel v0.3.29
   Compiling unicode-segmentation v1.10.1
   Compiling askama_escape v0.10.3
   Compiling crossbeam-queue v0.3.8
   Compiling rustls v0.21.9
   Compiling heck v0.4.1
   Compiling tokio v1.34.0
   Compiling zeroize v1.7.0
   Compiling spin v0.9.8
   Compiling tracing-attributes v0.1.27
   Compiling uniffi_checksum_derive v0.23.0 (https://github.com/mozilla/uniffi-rs.git?rev=22b8c21f117faa95c6d398baad463ef7f108749e#22b8c21f)
   Compiling byteorder v1.5.0
   Compiling crypto-common v0.1.6
   Compiling block-buffer v0.10.4
   Compiling block-padding v0.3.3
   Compiling untrusted v0.9.0
   Compiling tinyvec_macros v0.1.1
   Compiling digest v0.10.7
   Compiling inout v0.1.3
   Compiling tinyvec v1.6.0
   Compiling hmac v0.12.1
   Compiling cipher v0.4.4
   Compiling pkg-config v0.3.27
   Compiling opaque-debug v0.3.0
   Compiling zerocopy v0.7.26
   Compiling futures-sink v0.3.29
   Compiling vcpkg v0.2.15
   Compiling toml v0.5.11
   Compiling askama_shared v0.12.2
   Compiling cargo-platform v0.1.5
   Compiling libsqlite3-sys v0.26.0
   Compiling hkdf v0.12.3
   Compiling unicode-normalization v0.1.22
   Compiling cargo_metadata v0.15.4
   Compiling ff v0.13.0
   Compiling unicode-bidi v0.3.13
   Compiling smallvec v1.11.2
   Compiling allocator-api2 v0.2.16
   Compiling hashbrown v0.14.2
   Compiling group v0.13.0
   Compiling askama_derive v0.11.2
   Compiling rustls-webpki v0.101.7
   Compiling either v1.9.0
   Compiling sha2 v0.10.8
   Compiling universal-hash v0.5.1
   Compiling der v0.7.8
   Compiling percent-encoding v2.3.0
   Compiling base16ct v0.2.0
   Compiling static_assertions v1.1.0
   Compiling pin-utils v0.1.0
   Compiling uniffi_core v0.23.0 (https://github.com/mozilla/uniffi-rs.git?rev=22b8c21f117faa95c6d398baad463ef7f108749e#22b8c21f)
   Compiling sec1 v0.7.3
   Compiling form_urlencoded v1.2.0
   Compiling itertools v0.11.0
   Compiling parking_lot v0.12.1
   Compiling idna v0.4.0
   Compiling sct v0.7.1
   Compiling crypto-bigint v0.5.5
   Compiling scroll v0.11.0
   Compiling tracing-core v0.1.32
   Compiling iana-time-zone v0.1.58
   Compiling equivalent v1.0.1
   Compiling unicode_categories v0.1.1
   Compiling plain v0.2.3
   Compiling crc-catalog v2.4.0
   Compiling base64 v0.21.5
   Compiling siphasher v0.3.11
   Compiling crc v3.0.1
   Compiling goblin v0.6.1
   Compiling rustls-pemfile v1.0.4
   Compiling uniffi_meta v0.23.0 (https://github.com/mozilla/uniffi-rs.git?rev=22b8c21f117faa95c6d398baad463ef7f108749e#22b8c21f)
   Compiling elliptic-curve v0.13.8
   Compiling sqlformat v0.2.2
   Compiling tracing v0.1.40
   Compiling chrono v0.4.31
   Compiling indexmap v2.1.0
   Compiling tokio-stream v0.1.14
   Compiling url v2.4.1
   Compiling futures-intrusive v0.5.0
   Compiling uniffi_testing v0.23.0 (https://github.com/mozilla/uniffi-rs.git?rev=22b8c21f117faa95c6d398baad463ef7f108749e#22b8c21f)
   Compiling atoi v2.0.0
   Compiling webpki-roots v0.24.0
   Compiling askama v0.11.1
   Compiling hashlink v0.8.4
   Compiling digest v0.9.0
   Compiling weedle2 v4.0.0 (https://github.com/mozilla/uniffi-rs.git?rev=22b8c21f117faa95c6d398baad463ef7f108749e#22b8c21f)
   Compiling android_system_properties v0.1.5
   Compiling dotenvy v0.15.7
   Compiling bitflags v2.4.1
   Compiling ppv-lite86 v0.2.17
   Compiling rustix v0.38.25
   Compiling event-listener v2.5.3
   Compiling glob v0.3.1
   Compiling hex v0.4.3
   Compiling uniffi_bindgen v0.23.0 (https://github.com/mozilla/uniffi-rs.git?rev=22b8c21f117faa95c6d398baad463ef7f108749e#22b8c21f)
   Compiling sqlx-core v0.7.2
   Compiling rand_chacha v0.3.1
   Compiling rfc6979 v0.4.0
   Compiling signature v2.2.0
   Compiling async-compat v0.2.3
   Compiling aead v0.5.2
   Compiling include_dir_macros v0.7.3
   Compiling proc-macro-error-attr v1.0.4
   Compiling android-tzdata v0.1.1
   Compiling rand_core v0.5.1
   Compiling linux-raw-sys v0.4.11
   Compiling finl_unicode v1.2.0
   Compiling stringprep v0.1.4
   Compiling curve25519-dalek v3.2.0
   Compiling ecdsa v0.16.9
   Compiling rand v0.8.5
   Compiling futures-executor v0.3.29
   Compiling sha1 v0.10.6
   Compiling poly1305 v0.8.0
   Compiling flume v0.11.0
   Compiling md-5 v0.10.6
   Compiling proc-macro-error v1.0.4
   Compiling home v0.5.5
   Compiling whoami v1.4.1
   Compiling fastrand v2.0.1
   Compiling sqlx-sqlite v0.7.2
   Compiling sqlx-postgres v0.7.2
   Compiling tempfile v3.8.1
   Compiling uniffi_build v0.23.0 (https://github.com/mozilla/uniffi-rs.git?rev=22b8c21f117faa95c6d398baad463ef7f108749e#22b8c21f)
   Compiling primeorder v0.13.6
   Compiling polyval v0.6.1
   Compiling salsa20 v0.10.2
   Compiling chacha20 v0.9.1
   Compiling block-buffer v0.9.0
   Compiling indexmap v1.9.3
   Compiling signature v1.6.4
   Compiling base64ct v1.6.0
   Compiling ed25519 v1.5.3
   Compiling sha2 v0.9.9
   Compiling password-hash v0.5.0
   Compiling sqlx-macros-core v0.7.2
   Compiling chacha20poly1305 v0.10.1
   Compiling xsalsa20poly1305 v0.9.1
   Compiling ghash v0.5.0
   Compiling x25519-dalek v1.1.1
   Compiling include_dir v0.7.3
   Compiling ctr v0.9.2
   Compiling aes v0.8.3
   Compiling blake2 v0.10.6
   Compiling aho-corasick v1.1.2
   Compiling atty v0.2.14
   Compiling hashbrown v0.12.3
   Compiling regex-syntax v0.8.2
   Compiling os_str_bytes v6.6.1
   Compiling termcolor v1.4.0
   Compiling clap_lex v0.2.4
   Compiling argon2 v0.5.2
   Compiling sqlx-macros v0.7.2
   Compiling aes-gcm v0.10.3
   Compiling regex-automata v0.4.3
   Compiling uniffi-kotlin-multiplatform v0.1.0 (/Users/dkulic/work/comunity/indicio/aries-askar/uniffi-kotlin-multiplatform-bindings)
   Compiling crypto_box v0.8.2
   Compiling clap_derive v3.2.25
   Compiling ed25519-dalek v1.0.1
   Compiling p384 v0.13.0
   Compiling p256 v0.13.2
   Compiling k256 v0.13.2
warning: unused variable: `name`
   --> uniffi-kotlin-multiplatform-bindings/src/gen_kotlin_multiplatform/mod.rs:414:30
    |
414 |             Type::External { name, crate_name, kind } => {
    |                              ^^^^ help: try ignoring the field: `name: _`
    |
    = note: `#[warn(unused_variables)]` on by default

warning: unused variable: `crate_name`
   --> uniffi-kotlin-multiplatform-bindings/src/gen_kotlin_multiplatform/mod.rs:414:36
    |
414 |             Type::External { name, crate_name, kind } => {
    |                                    ^^^^^^^^^^ help: try ignoring the field: `crate_name: _`

warning: unused variable: `kind`
   --> uniffi-kotlin-multiplatform-bindings/src/gen_kotlin_multiplatform/mod.rs:414:48
    |
414 |             Type::External { name, crate_name, kind } => {
    |                                                ^^^^ help: try ignoring the field: `kind: _`

warning: field `inner_type_name` is never read
   --> uniffi-kotlin-multiplatform-bindings/src/gen_kotlin_multiplatform/mod.rs:229:5
    |
227 | pub struct SequenceTemplateCommon {
    |            ---------------------- field in this struct
228 |     ffi_converter_name: String,
229 |     inner_type_name: String,
    |     ^^^^^^^^^^^^^^^
    |
    = note: `#[warn(dead_code)]` on by default

warning: field `config` is never read
   --> uniffi-kotlin-multiplatform-bindings/src/gen_kotlin_multiplatform/mod.rs:30:13
    |
30  |               config: Config,
    |               ^^^^^^
...
285 | / kotlin_template!(
286 | |     AsyncTypesTemplateCommon,
    | |     ------------------------ field in this struct
287 | |     "common/AsyncTypesTemplate.kt.j2"
288 | | );
    | |_- in this macro invocation
    |
    = note: this warning originates in the macro `kotlin_template` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: field `config` is never read
   --> uniffi-kotlin-multiplatform-bindings/src/gen_kotlin_multiplatform/mod.rs:30:13
    |
30  |               config: Config,
    |               ^^^^^^
...
289 | / kotlin_template!(
290 | |     TopLevelFunctionsTemplateCommon,
    | |     ------------------------------- field in this struct
291 | |     "common/TopLevelFunctionsTemplate.kt.j2"
292 | | );
    | |_- in this macro invocation
    |
    = note: this warning originates in the macro `kotlin_template` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: field `config` is never read
   --> uniffi-kotlin-multiplatform-bindings/src/gen_kotlin_multiplatform/mod.rs:30:13
    |
30  |             config: Config,
    |             ^^^^^^
...
293 | kotlin_template!(UniFFILibTemplateCommon, "common/UniFFILibTemplate.kt.j2");
    | ---------------------------------------------------------------------------
    | |                |
    | |                field in this struct
    | in this macro invocation
    |
    = note: this warning originates in the macro `kotlin_template` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: field `config` is never read
   --> uniffi-kotlin-multiplatform-bindings/src/gen_kotlin_multiplatform/mod.rs:30:13
    |
30  |               config: Config,
    |               ^^^^^^
...
303 | / kotlin_template!(
304 | |     AsyncTypesTemplateJvm,
    | |     --------------------- field in this struct
305 | |     "jvm/AsyncTypesTemplate.kt.j2"
306 | | );
    | |_- in this macro invocation
    |
    = note: this warning originates in the macro `kotlin_template` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: field `config` is never read
   --> uniffi-kotlin-multiplatform-bindings/src/gen_kotlin_multiplatform/mod.rs:30:13
    |
30  |               config: Config,
    |               ^^^^^^
...
307 | / kotlin_template!(
308 | |     RustBufferTemplateJvm,
    | |     --------------------- field in this struct
309 | |     "jvm/RustBufferTemplate.kt.j2"
310 | | );
    | |_- in this macro invocation
    |
    = note: this warning originates in the macro `kotlin_template` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: fields `cbi`, `type_name`, and `ffi_converter_name` are never read
   --> uniffi-kotlin-multiplatform-bindings/src/gen_kotlin_multiplatform/mod.rs:69:13
    |
69  |               cbi: &'cbi CallbackInterface,
    |               ^^^
70  |               type_name: String,
    |               ^^^^^^^^^
71  |               foreign_callback_name: String,
72  |               ffi_converter_name: String,
    |               ^^^^^^^^^^^^^^^^^^
...
315 | / kotlin_callback_interface_template!(
316 | |     CallbackInterfaceTemplateJvm,
    | |     ---------------------------- fields in this struct
317 | |     "jvm/CallbackInterfaceTemplate.kt.j2"
318 | | );
    | | -
    | |_|
    | |_in this macro invocation
    | |_in this macro invocation
    |   in this macro invocation
    |
    = note: this warning originates in the macro `kotlin_callback_interface_template` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: fields `cbi`, `type_name`, and `ffi_converter_name` are never read
   --> uniffi-kotlin-multiplatform-bindings/src/gen_kotlin_multiplatform/mod.rs:69:13
    |
69  |               cbi: &'cbi CallbackInterface,
    |               ^^^
70  |               type_name: String,
    |               ^^^^^^^^^
71  |               foreign_callback_name: String,
72  |               ffi_converter_name: String,
    |               ^^^^^^^^^^^^^^^^^^
...
339 | / kotlin_callback_interface_template!(
340 | |     CallbackInterfaceTemplateNative,
    | |     ------------------------------- fields in this struct
341 | |     "native/CallbackInterfaceTemplate.kt.j2"
342 | | );
    | | -
    | |_|
    | |_in this macro invocation
    | |_in this macro invocation
    |   in this macro invocation
    |
    = note: this warning originates in the macro `kotlin_callback_interface_template` (in Nightly builds, run with -Z macro-backtrace for more info)

   Compiling rmp v0.8.12
   Compiling bls12_381 v0.8.0
   Compiling serde-json-core v0.5.1
   Compiling bincode v1.3.3
error: failed to build archive: No such file or directory

   Compiling cbc v0.1.2
warning: `uniffi-kotlin-multiplatform` (lib) generated 11 warnings
error: could not compile `uniffi-kotlin-multiplatform` (lib) due to previous error; 11 warnings emitted
warning: build failed, waiting for other jobs to finish...