matrix-org / matrix.org

matrix.org public website
Other
422 stars 348 forks source link

Suggestion: add a pointer to libolm on https://matrix.org/sdks/ (detailed suggestion inside) #931

Open ell1e opened 3 years ago

ell1e commented 3 years ago

Is your feature request related to a problem? Please describe. I was looking at https://matrix.org/sdks/ since I'm contemplating implementing a matrix client. However, I don't use any of the languages listed, so I was looking for something with a C API. Since there is no such entry, my temporary conclusion was "oh so I guess I'd have to hand-roll E2EE, there goes the idea of ever getting this practically done." But that conclusion was wrong, since there is libolm. So while that isn't the equivalent of a full Matrix SDK, I think having it in the "Other" section as a "Roll your own with libolm (C)" entry might be a great pointer.

Describe the solution you'd like I am suggesting to add an entry to the "Other" section of the Matrix SDK list at the bottom, named "Roll your own with libolm (C)". It could have the following description: "If you want to implement a Matrix client using another programming language than the ones with existing SDKs, or want to make a new SDK for your language, you can still use libolm to help you out if your language enables access to C libs via FFI. See here for more details: https://matrix.org/docs/guides/end-to-end-encryption-implementation-guide This should make a full implementation with E2EE support more feasible even without the use of a pre-existing Matrix SDK."

Describe alternatives you've considered I mean the info is there, just not on that SDK page but rather: https://matrix.org/docs/guides/end-to-end-encryption-implementation-guide So I guess it was mostly on me for failing Google. However, such a pointer to libolm might be very helpful for people to realize they may not need to hand-roll the E2EE part entirely just because there is no Matrix SDK for their target language, so I think it could be of great value.

Additional context

ell1e commented 3 years ago

Actually on second thought, maybe there should just be a complete matrix client SDK for C? I was told in the chat "well HTTP(S) is easy so libolm does the hard part" and while that might be true, it's still not not time-consuming to reimplement this all manually and I don't think it is particularly constructive for client devs as a whole. It just limits how many people are willing to jump in making a Matrix client outside of the "approved languages", while making an FFI wrapper for an existing C lib is usually way more doable for knowledgeable programmers in an unusual language than rewriting significant chunks of software from scratch just to have it available in that language. Consequently, a C SDK could optionally be used in place of the many others for other langs with wrappers (due to how many langs have good C interop) and actually reduce the overall amount of duplicate effort of implementing the Matrix protocol for those who feel up to just working with a C wrapper instead. I think that'd be a great option to have, at least.

Any ideas on where I could make a ticket to suggest a C API matrix client SDK?

poljar commented 3 years ago

A complete SDK in C would be a tough order. I think a better approach is to take the Rust SDK and either cbindgen or cxx to generate C/C++ bindings.

The Rust SDK also has a modular structure where the e.g. crypto layer is completely separate and can be plugged into another library in a completely different language separately, the same can be done with the base layer which handles state storage.

This makes it possible to pick the layer you would like to plug into your client library. For example if you're only lacking crypto support pick the crypto layer, if your library is a stateless wrapper around the Matrix HTTP API pick the base layer, if you don't have a library at all pick the full SDK.

There are plans to plug the crypto layer into Element Android and to plug the base layer into matrix-nio so such use-cases for other languages will be supported.

ell1e commented 3 years ago

@poljar rust is not particularly suited for C bindings because it has different OOM behavior that is a bad match in the C world: https://github.com/hyperium/hyper/issues/2265#issuecomment-694043499 so I think wrapping any of the SDKs that actually has proper OOM handling might be a better match, personally. Rust is also less portable compared to C if e.g. trying to cross compile it for more obscure platforms. Rustc is also a resource hog which can make building in some environments considerably harder. I would probably not be interested in a wrapped Rust SDK personally for all these reasons, not sure if I'm the only person. Of course, it's still better than nothing so maybe it's "good enough" for many client devs, I don't know, these are just my personal thoughts.

poljar commented 3 years ago

I'm not aware of any SDK that doesn't abort when it fails to allocate memory and there are plenty C libraries that do abort when allocating memory fails. Are there any?

Looking at the list of supported platforms I would argue that Rust is portable enough.

While it is true that compiling Rust projects can be a challenge on less powerful hardware, and certainly the Rust SDK will need to be improved with regards to compile times, cross compiling works very nicely with Rust and is one way to avoid this problem.

I wouldn't want to discourage anyone from writing a new SDK in C, but this is a daunting task in higher level languages on it's own. Considering that the Rust SDK already provides a way to avoid duplicated efforts, before anyone writes yet another SDK in C, at least consider the Rust SDK.

ell1e commented 3 years ago

and there are plenty C libraries that do abort when allocating memory fails

There are plenty that also do not, from those I've ever used it's the majority. This is particularly a problem when creating and offering more intermediate tooling instead of being the "final application author", because this "poisons" all the intermediate tooling with the bad behavior as well. Although for something as highlevel as a Matrix SDK, not handling OOM nicely is probably more acceptable.

Looking at the list of supported platforms I would argue that Rust is portable enough.

No M1 Macs, no ARM64 Windows, no MIPS (which is used in some retro handhelds where it's not completely absurd to want a Matrix client since some got raspberry 2 like specs and wifi), ... I'd say that actually doesn't look very good at all. It is worse than I thought, if I'm being honest. C would work no problem on any of these, if written portably enough (which most C is). Oops, only looked at Tier 1. Looks pretty good, actually, my bad.