[Assessment/Research Spike] Simplifying Cocoapods build and Distributing over SPM

[pacu]

@pacu has already done a Research Spike on this, but we would like more input.

• How can we make the build process easier?
• Can we achieve a Rust-less build, where the developer does not need to care about rust dev environment to just be a mere user of this library?
• How can we distribute this over Swift Package Manager?

[dh-ecc]

How can we make the build process easier?

(some ideas, as already aluded to)

• Rust-less build for SDK consumer
• Rust-less build for SDK contributor if not changing rust source
• Rust-less build for SDK contributor with ability to update binary rust source
• Ability for SDK contributor to work simultaneously on rust and swift source

Can we achieve a Rust-less build, where the developer does not need to care about rust dev environment to just be a mere user of this library?

yup 😊 See below

How can we distribute this over Swift Package Manager?

yup 😊 See below.

---

Initial Results

Rust-less build

It is possible to have a Rust-less build for the consumer of the SDK by packaging librustzcashlc as a binary, or more specifically an XCFramework to support multiple platform options. side-note The same process, without the packaging as an xcframework, and + a few extras could be used to provide kotlin/native bindings as well (I made a small exploration but I'm not too familiar with it, will be good to run it past android for feedback).

SPM

With the creation of a librustzcashlc xcframework, this means it can then be included as a binary dependency for a swift package, since the rest of ZcashLightClientKit is all swift source, these can trivially be part of a swift package too. We can therefore have a purely SPM based distribution 🙌. A demonstration of this with a small readme can be found at dh-ecc/ZcashRustLib.xcframework.

As a consumer of the SDK, the spm integration works out quite nicely 😄 (I also threw in macOS support) check out dh-ecc/PrintBlockHeight

Additional Considerations

Adding SPM support will greatly improve the consumer UX, it will have the added bonus of removing the rust dependancy and all script integrations from the cocoapods distribution.

It also gives contributors a nice SPM based developer workflow and can be extended with an additional workspace to build the rust target as a dependency of the swift library, this would require a rust toolchain, but it would be exclusively if you wanted to modify the rust portion of the code.

Pre built rust binary also gives us a great CI time saving since we only need functioning swift toolchain and don't have to setup a rust one.

[pacu]

Some dude, Hasan Ali Atak https://github.com/haznai, emailed me asking about #296 research spike and sent me this link https://github.com/mozilla/rust-components-swift I haven't looked at it deeply yet. But looks interesting

[pacu]

@dh-ecc How would this play with Cocoapods specifically? Many developers that could potentially adopt Zcash on their apps still use it a dependency manager I do think we still need to support it.

[str4d]

The same process, without the packaging as an xcframework, and + a few extras could be used to provide kotlin/native bindings as well (I made a small exploration but I'm not too familiar with it, will be good to run it past android for feedback).

I've separately been looking at using uniffi to simplify the production of the Rust component (and reduce the duplicated FFI work between the Swift and Kotlin sides). I think the performance cost would not matter too much (the heavy work occurs on the Rust side), would be outweighed by the reduced maintenance surface (and Mozilla upstream are incentivised to improve it over time), and we could also then use that for e.g. Python bindings.

We could do a straightforward translation of the current Swift and Kotlin FFI APIs to begin with, but I'm interested in leveraging uniffi to construct a better API.

[str4d]

My initial experiment from a month or so ago: https://github.com/str4d/librustzcash/tree/zcash_client_sqlite_ffi/zcash_client_sqlite/ffi

[dh-ecc]

[@pacu]

Some dude, Hasan Ali Atak https://github.com/haznai, emailed me asking about #296 research spike and sent me this link https://github.com/mozilla/rust-components-swift I haven't looked at it deeply yet. But looks interesting

It does! This was referenced in mozilla/uniffi-rs (transitively via) https://github.com/mozilla/application-services. Seems like it could be a really good reference. Good find 👍!

[@pacu]

@dh-ecc How would this play with Cocoapods specifically? Many developers that could potentially adopt Zcash on their apps still use it a dependency manager I do think we still need to support it.

Oh 💯 I definitely don't suggest dropping CocoaPods support. This actually ends up making CocoaPods support a lot easier as we can ditch the build script phase and use only vendored framework and sources on the podspec, I'll push up a commit with this if I can get some time today to dh-ecc/ZcashRustLib.xcframework

[@str4d]

We could do a straightforward translation of the current Swift and Kotlin FFI APIs to begin with, but I'm interested in leveraging uniffi to construct a better API.

I'm definitely interested in this approach too, and we can indeed do this incrementally, first with C bindings then transitioning to uniffi 🙌

[@str4d]

My initial experiment from a month or so ago: https://github.com/str4d/librustzcash/tree/zcash_client_sqlite_ffi/zcash_client_sqlite/ffi

Awesome! I'll check this out 👍

[str4d]

To anyone looking at this issue 2.5 years later, this Rust-less build has created significant problems for the Rust FFI development process, as well as greatly reducing the agility of the Swift SDK in response to Rust improvements. An alternative approach is being developed here: https://github.com/Electric-Coin-Company/zcash-light-client-ffi/issues/96