Design decisions - Why are UniFFI generated types wrapped in hand rolled types?

[theIDinside]

I have a few questions regarding the use of the rust-sdk; and I keep seeing that so much of the UniFFI generated types are wrapped in hand rolled Kotlin types, which makes me wonder - why? Doesn't this defeat the purpose?

• 1: it means you now have to handroll everything in the rust-sdk which defeats the re-use argument, as you now have to handwrite everything anyhow; yes, without the implementation details of the Java Native Interface (JNI), but you've still added yet another layer on top of what UniFFI does so well.
• 2: It also defeats other benefits of UniFFI which is rapid adoption; since you're now wrapping the generated types in interfaces and implementing classes as well (!).

Is there something I'm missing? Because I'm seeing it everywhere - from MatrixClient to RustMatrixClient to Room to RustRoom to MatrixUser (called UserProfile in Rust, and also called UserProfile in the generated FFI type) and even down to the record types; essentially tripling the complexity - because, on top of having the UniFFI created types, you now have an interface and an implementation class also.

[jmartinesp]

There are several reasons for this, although I agree some duplications could be avoided:

1. Anything that's not a pure record type and comes from Rust will be a memory leak unless you call its close() or destroy() methods, so most of the time we map those to our own objects in a .use { ... } block that automatically releases them.
2. When we wrap them in another class, we can also improve input/output and error handling.
3. UniFFI does produce interfaces for each class that's not a pure record type, but if you use these classes as parameters in functions or callbacks, the generated Kotlin code will use the class instead of the interface, so you need to pass the originally retrieved object instead of an abstraction of it. When we either wrap these objects, we can just mock/disable these function calls by adding an alternative or no-op implementation:

// Example of what the FFI generated code looks like, this is not real code
interface RoomInterface { ... }

class Room : RoomInterface { ... }

interface Client {
  // Not the interface, needs a real Rust object retrieved from Rust, that can become a memory leak
  fun joinRoom(room: Room)
}

If points 1 and 3 could be fixed then we could probably just use these objects right away, but I don't think they'll be fixed any time soon.

[theIDinside]

Right, I understand that as well, and I just found out that for example SlidingSyncRoom is not long lived on the Rust side (where the Kotlin side safely could hold on to that object for as long as it wanted to, and clean up a collection of those objects at shutdown), which was strange to me, as that was the behavior I would expect. But I realize now, that it's (the Rust side) not that much of an actual client, and more of an interface of communication using the matrix protocol.

That answers my question, but I've thought of another one;

I'm starting to think, that maybe it would be better to write most of the client code directly in Rust? i.e. setting up listeners, processing of the data & client state (like applying diffs to the timeline, etc), all that, and then just have the Kotlin side be thin client that merely gets processed/finalized-for-display output from the Rust side? Wouldn't that solve all issues wrt life times and such since the only data passing the FFI-barrier would be "pure" data - or is this just not possible on current state of FFI on the JVM platform? Meaning, the Kotlin side would just represent UI, not actual client state? Or would this not be feasible (maybe it's even outside the scope of this project)?

Edit: I actually am starting to think this is actually a better idea than I had first expected. Right now, the uniffi is exposing things that aren't optimal (and never can be, due to it having to be as general as possible; being able to support any arbitrary front end client). The crypto stuff etc, all that is handy dandy to have written in Rust, being performant, reusable, etc. But handrolling a more custom "Rust client" exposed to Kotlin via more fine tuned control, seems like a much better idea, skipping over entirely the mounting life time issues and JVM:s sub-par FFI, while at the same time gaining all the benefits of performant "backend" code that no kotlin implementation could ever compete with. But this is my naivety speaking and I might not be seeing the full picture.

In this world, I am seeing the setup as a sort of REST api, as it were, where the kotlin side can "post" "requests" to the rust backend (i.e. function calls using JNI), and the Rust backend serving pure data as response which the Kotlin side can take full ownership over, garbage collection and the whole shebang. Or would such a solution be impossible with JVM-FFI?

[jmartinesp]

I'm starting to think, that maybe it would be better to write most of the client code directly in Rust?

I think that's where we're heading to, but keep in mind both the Rust SDK and Element X are in quite early stages. In example, sliding sync is mostly managed right now in the platforms (Android/iOS) side but the Rust SDK team is internally working on handling all the hard bits inside the SDK, simplifying its usages as much as possible.

[jmartinesp]

Closing this issue as both questions have been answered and we're little by little moving most logic code to the Rust SDK. There has been some movement in UniFFi about memory handling and using interfaces, let's hope the discussion materializes and lets us improve our usage of the FFI bindings 🤞 .