search

y-crdt / yswift

Swift language bindings to Y-CRDT
https://y-crdt.github.io/yswift/documentation/yswift/
Other
66 stars 6 forks source link

Consumption goal for the library #8

Closed heckj closed 1 year ago

heckj commented 1 year ago

I'd like to see the end result of what we create here be usable through Swift Package Manager - the idea being that someone could add a dependency to "Y-CRDT" (or whatever we call the module), and have that automatically pull in the dependencies that we create and make available from this repo - the swift overlay and the underlying static C libraries.

The target for something like this would be to allow a developer to add a dependency to their Package.swift file:

dependencies: [
        .package(url: "https://github.com/y-crdt/y-uniffi", from: "0.1.0"),
],

and from there be able to add an import that makes YDoc available in the code:

We might even want to think about what we want the name of this module to be in terms of swift import - so from the call sites you do something akin to:

```swift
import YCRDT

let doc = YDoc()

Along these lines, what do we want the various names to be that are used here? There are three we need to nail down:

1) repository to use for the C library/XCFramework dependency (currently y-uniffi) 2) a repository for the swift overlay dependency (also currently y-uniffi, but only because it's all in the same place - I don't know how possible it is to have two different dependencies served from the same repository) 3) The module name that a developers imports to use the library (currently YSwift - but I'd like to suggest YCRDT instead)

nugmanoff commented 1 year ago

I was thinking about good naming for this Zoo of dependencies and packages, but I don't have anything good to propose for now. I will post to this thread once I have something.

Regarding the consumption goal: if I am getting it right, the way you described it (just adding dependency to Package.swift file) is exactly the way it works now:

https://github.com/y-crdt/y-uniffi/blob/main/yswift/Tuist/Dependencies.swift#L4-L6

The only difference being that we are importing by the path right now, but it will change once we split y-uniffi to the separate repo.

Maybe Tuist/Dependencies.swift file confuses things. But it basically is an abstraction over Package.swift, Cartfile (for Carthage) manifest files.

E.g. I can create a new vanilla project and just add the dependency from lib folder by the path. Note that I am saying dependency, because lib/ folder contains SPM Manifest file lib/Package.swift

Let me know if I am missing on something here

heckj commented 1 year ago

That's definitely the way it's working now, with Tuist specifically - but the primary difference there that what's set up only works with a local library reference. That's perfect for where we are: learning, experimenting, and supports very quickly iterating on changes.

What the current setup doesn't yet do is support external, versioned releases - for which I'm noodling nailing down and trying a process to make a release so that it can be used externally - and that same setup can be used to host documentation with the Swift Package Index, with multiple versions tracked based on major version, etc. I think this was a goal all along, I just wanted to spell it out to make sure. And to start the discussion around the soup of names, as you noted.

I don't have a strong opinion about the internal names for the C FFI library - The double Y prefix works perfectly fine for me, but I would like think a bit more explicitly about the type names as they're exposed in Swift. The primary reason there is Swift has no real namespace control, and Y-CRDT is already using common terms for its internal components. This wouldn't be any sort of issue at all if there was better namespacing, but alas - it's the world that is.

To illustrate, let me present a hypothetical setup. Within the Y-CRDT swift module, if a versioned branch of the CRDT has a data structure named View, and you were working with another module that also had View (for example, SwiftUI), then in the code it would far less than clear which view type you were referring to. As a general pattern, quite a number of Swift libraries (mostly from the Objective-C days with a similar problem space) prefix their most common terms to make the type names unique, and to some extent Yjs has already been doing that - and I think we definitely want to continue that setup.

We've got YDocument, YProtocol, and YArray already, and I think it makes good sense to stick with those type names to make it clear they're the wrapped Y-CRDT types (and extend that to YMap when I "get off my duff" and dive into that bit of the Rust/UniFFI setup).

heckj commented 1 year ago

After bungling adding macOS support to the XCFramework, backing it out, and adding CI to try and save myself from future missteps, I think I've got a path forward. There are a few key constraints here though, so the output of this effort could change how and where we're doing things in this repo.

First off - the current setup with build-xcframework.sh and tuist ends up creating a chain of 3 module targets, and another for the swift example iOS app code:

YniffiXC <- Yniffi <- YSwift <- iOS Example code

The first two are build by build-xcframework.sh in the lib directory:

The last (YSwift) is formed by Package.swift in the yswift directory of this repository.

(My earlier mistakes and confused were in incorrectly assuming there was only a single swift package that used the associated C FFI)

Using uniffi ends up setting a constraint: the generated swift scaffolding is tightly coupled to the headers and static libraries in terms of specific symbol names. In practice, this means that any time we iterate on the underlying dependencies such that we need to rebuild the static library, we'll want to use the associated swift code that the uniffi process generates as well.

Another constraint comes from Swift itself: In order to use a repository as a remote package reference, the Package.swift is expected to reside in the root directory. The mono-repo setup of two swift packages (and associated example code) doesn't easily allow for an external developer to be able to add the package dependency and get the related binary bits that we release.

I'm also taking our fast iteration using all local packages as an exceedingly useful setup, and want to preserve that setup so that we can continue to iterate on exploring the swift language binding construction without being forced to do a bunch of explicit releases.

Based on all that, I'd like to propose the following:

When we get to point where we want to release this code for use to external swift developers, we would need to maintain a coordinated, coupled release of both the .xcframework binary target and the associated swift package YCRDT, updating the Package.swift in the root repository from a local XCFramework to using a remote XCFramework that we reference and host from Github as a release. The process for that release would end up needing to look something like:

Alternatives:

  1. We could continue to keep the separate swift packages (Yniffi and YSwift), with Yniffi only containing the code generated from uniffi-bindgen. We could still move this work (effectively, what's in the lib directory) to the root of the repository so that external packages can depend on it. Then migrate the developer-generated additional swift code (such as YArray, YDoc, etc - the stuff in the YSwift module today) into a separate repository. I still think it would be valuable to rename the module generated from YSwift to YCRDT to reflect that a developer is importing the language bindings that they can then directly use.

The primary benefit of this alternative would be to separate the release process of the rust-based code (and any dependency updates) from the swift-based code.

nugmanoff commented 1 year ago

On naming:

The reason that I am preferring YSwift – because it follows the pattern of yjs, yrs, ywasm, yrb, ypy and it is kind of natural to assume that Swift binding follows the same pattern. Eventually all of that projects fall under Y organization. Which is kind of cool umbrella-unification. Historically GitHub organization that current repository reside in was called y-crdt, and I think it is also a good name something that serves umbrella purpose, as well as something like y-org.

I still think it would be valuable to rename the module generated from YSwift to YCRDT to reflect that a developer is importing the language bindings that they can then directly use.

I honestly think that importing YSwift reflects intention of Swift language binding to Y-thing better than YCRDT, which feels more generic and might imply that you are importing something more low-level than Swift language binding.

On packaging

Intuition that I have packaging could be phrased as – optimize for simplicity and ease-of-use for now, until we cover core functionality and be ready to cut v1 release.

Basically I am for moving everything uniffi-related to separate repository, which will produce artifacts for Swift and Kotlin as part of its release process. If I am getting it right, it is what you've suggested as Alternative approach.

So there would be 3 repos (omit the names): y-uniffi, y-kotlin and y-swift:

And there would be 2 reasons to change & release the final package (YSwift/YCRDT) that will be consumed by users:

  1. Downstream changes from yrs. Something changes in yrs → we change that in y-uniffi and produce artifact → we release new version of the package (YSwift/YCRDT) with that updated artifact.
  2. Some improvements to language bindings itself. We change something in y-swift and we cut new release.

That's how I see it, happy to discuss more!

Btw, feels so great to have someone to discuss all these details with. Thanks Joe! Amazing attention to details, and very neat write-up here!

heckj commented 1 year ago

Sounds good. YSwift it is then! There's some generated code from the uniffi-bindgen that we'll need to continue to include in the y-uniffi repo - so that repo will need to have not only the C static libraries but also a small bit of swift in a Swift Package to go along with it. I haven't yet looked at anything equivalent in the Kotlin space to have any sense of how that platform manages its modules.