feat: rewrite in rust

[WcaleNieWolny]

First of all, I know how radical proposal it is. However, I think that it is something that we should consider.

Currently, this plugin is composed of a lot of features that have been designed in one point of time that may or may not work currently and that may or may not integrate well with different features.

There are A LOT of things that are in a semi-working, semi-broken state that would require a breaking release to get fixed. For example

• Encryption
  • Encryption currently relies on a very complex RSA + AES system that could just be simplified to AES.
  • Rust makes encryption attacks harder as it would allow to move the AES key from capacitor.config.ts into an executable making it much harder to retrieve. With further built time obfuscation it greatly increases the difficulty of getting the AES key
• Checksum
  • Currently the checksum is done AFTER encryption making it useless. It has no value as is.
  • It can be resolved without a breaking release by implementing a new checkum system while supporting the old one from the CLI, but that further increases the legacy code that we have to maintain.
• Bundle info
  • Bundle info has A LOT of unused methods
  • The encoding/decoding is done differently between platform with manual marshaling done in android
  • Methods are scattered in different places. Take a look at the bundleExists, getBundleInfo, method
  • The current UserDefaults/SharedPreferences does not include a version field making it impossible to add/remove fields from BundleInfo without breaking compatibility.
    • Furthermore, UserDefaults prevents fingerprinting which causes the UUID to be handled differently between IOS and Android.
• Download:
  • Currently the download method behaves very differently between IOS and Android with 2 district implementation for concurrency
  • Code logic is not the same between IOS and Android. IOS will NOT perform checks if a bundle has already been downloaded
  • Android handles all of the HTTP manually which is very distinct from the Alamofire lib used in IOS

Those are just some of the example that I could think of right now. There are very likely more problems that I forgot to mention.

The biggest reason for rewriting this is the potential introduction of partial update.

Martin has told me that he wishes to implement partial update V0 without making it a breaking release and release a breaking one once capacitor V7 is released. This likely is possible; however, it's likely that by the time that capacitor v7 is released we are going to have very little time to address all of those issues. It IS possible to resolve most of them them without a rewrite but with the amount of changes required it's very likely going to take quite a while

My motivation for this rust rewrite are as follows:

• Easier E&E testing of the plugin
• Easier integration of react native in the future
• Reducing legacy code
• Simplified progressive download development

• We are going to need a breaking release in one point or another

  I propose that we:

  • Develop a V0 version of progressive download with the current java/swift architecture
  • Start working on this rewrite to release a V7 of this plugin

  How I plan to structure this rewrite:

• Core (a rust library that does NOT depend on any platform specific code)
• rust-java (A rust <-> java bridge around the core library that utilizes JNI expose the core library safely to java)
• rust-swift (A rust <-> swift bridge around the core library that utilizes swift FFI to expose the core library safely to swift)
• ios-plugin (An IOS plugin that uses rust-swift in order to communicate with the rust library)

• android-plugin (An Android plugin that uses rust-java in order to communicate with the rust library)

  
  
  Here is why I think rust can help us a lot
  - testing
  - This a a HUGE deal. Implementing E&E tests and unit tests with this rust architecture is a peace of cake.
     - First, we can unit test individual functions of the rust `core` lib to ensure that they work exactly as designed
     - Second, we can write java/swift tests to test the bridge itself
     - Lastly, we can write a custom E&E testing framework that will utilize the emulator.
   - Unit testing is not quite possible due to how intertwined the capacitor dependent code is with the pure swift/java functions
   - End to end testing is not quite possible due to the lack of java macros. E2E would likely connect to some form of a TCP socket to transfer events into the E2E framework. The framework would await those events. This debug events should be hidden behind a feature flag. Java does not have support for feature flags as far as I know.

• SQLite for Bundle info and partial update manifest
  • This could help us resolve the fingerprinting problem with IOS
  • It would provide easy migrations for further updates to the system
  • It would make implementing partial update manifest easy
• Unified code base
  • changes would no longer have to be made in both IOS and Android impls decreasing development time

Here is why rust may be a terrible idea

• Rust is a HARD language.
  • Personally, I have worked for about 9-12 months in a rust project and I still consider it very hard
  • As far as I know Martin has no rust experience
  • It could be harder to find people to solve bounties
• There are no guarantees about libraries
  • We would need to use A LOT of rust libraries (crates). There is no guarantee that they would work in IOS/Android
  • Rust STD for Android/IOS is in stage 2.
• Time
  • This rewrite would likely take months
• Risk of crashes
  • Rust is a low level language. If we make a mistake in the FFI we risk crashing the user's application. Several factors can be introduced to prevent this happening but it still increases the risk

[riderx]

Why rust over c++, kotlin or others language who compile binary? Never ear about rust for mobile, but did for others so it’s not a good sign at all and will be my last choice if rewrite is decided

[riderx]

Please create one issue for each report you did that and link them in your original message

[WcaleNieWolny]

Why rust over c++, kotlin or others language who compile binary?

Kotlin native is a garbage collected language with experimental swift interop and no java interop as far as I can see (C interop is possible [but experimental] and from C we can do JNI but that is just terribly complicated). This alone should be enough to eliminate kotlin native

Both C and C++ are viable alternatives to rust but they I believe more in rust them in C or C++

Rust has:

• Excellent tooling

  • We are going to be supporting AT LEAST 6 targets
    • Android x86
    • Android ARM
    • IOS ARM
    • IOS ARM simulator
    • IOS X86 (simulator)
    • Native (depends on your pc, can be linux x86 or macos ARM) - used for unit tests
  • Good luck trying to compile GCC, setting up cmake to support this setup. Trust me on this one, make, setting up gcc for cross-compiling is HELL. I would not wish it on my worst enemy to have to set this up

• Excellent repository of libraries - crates.io

  • I have tried using a C++ testing library once. There are A MILION ways to set it up and you spent hours fighting with CMAKE to get it to work.
    • We need to compile a FAT binary. We cannot afford to do dynamic linking for anything. (Or at least we want to limit dynamic linking to the minimum). This is easier with rust
    • There are a number of crates we would use for HTTP, RSA, AES, GZIP, Swift FFI, Java JNI (FFI), SQLite, JSON, concurrency. The more libraries the more complex CMAKE you are going to need

• Memory safety

  • Have you ever wrote a C program where you accessed an array out of bounds causing a SegFault?
    • I have, trust me - it's not fun. We are going to be running this code on thousand of devices. We don't want to end up like crowdstrike
  • According to this article 24% of CVES are caused my memory management issues
  • Memory safety gets complicated with FFI but this can be addressed with those crates: java, swift.
    • The core library would be fully safe. The bridge would be the only place where unsafe is used and it would be managed by a library.

• The language itself

  • Do you KNOW how to use C++? I don't. I know C, but that likely is a bad idea due to the mentioned memory safety
    • I did work for 9 months building a performance critical rust app

[riderx]

• I have been working professionally in c++
• Any work outside Native language will be a shitshow.
• Java and Swift bridge for Rust never reached a major version, not a sign of maturity. Even if they exist since long.
• Language without garbage collection is the opposite of the value I follow in Capgo.
• Changing language, make the community participation go to 0 as most are web dev, and do use a bit of swift and Java
• Rewrite all and in another language, it's a tech base decision and 0% related to business. A strategy we cannot afford.

Rewrite to another language is out of scope.

Learning from this discussion. I think we can write abstraction to separate, Capacitor stuff to Capgo stuff, and in the future make it easy to bridge, if we find a better way, mature enough, fitting Capgo philosophy.