Custom Content Types

[nplasterer]

It would be nice to be able to create custom content types in react native like we do in the other sdks.

[dmccartney]

High level, supporting custom content types in RN looks a little like this:

• [x] support implementing the custom codecs in iOS and Android (done w/ existing SDKs)
• [ ] support registration of custom codecs (iOS and Android) with RN XMTP native module This will take some careful API design (it's nonobvious because it has to work nicely across JS and native contexts and RN build quirks)
• [ ] support defining how the decoded content (or references to it) serialize across the RN/JS bridge (sending and receiving) We'll need to encourage this definition to be careful not to overwhelm the RN/JS bridge. This is the current system:
  • defining the JS types
  • teaching the XMTP RN native modules (iOS and Android) how to serialize the decoded message to/from the JS type
  • adding any ancillary native methods required (e.g. we added decryptAttachment and encryptAttachment to deal w/ remote attachments using file-references instead of sending the large file contents across the JS/Native bridge)

[neekolas]

I know there was a discussion about this earlier that I missed, but why not just ship bytes out of iOS/Kotlin and handle all encoding/decoding in JS?

[dmccartney]

I know there was a discussion about this earlier that I missed, but why not just ship bytes out of iOS/Kotlin and handle all encoding/decoding in JS?

The short answer is for performance. I sometimes forget that content types doesn't just mean "a little meta data on a reply" -- it also means large files with complicated encryption and rendering schemes.

• The JS in RN is slow at crypto/byte-data work (single-threaded, unreliably TypedArray implementations, etc) -- if not for this we could try to just use xmtp-js for everything.
• The JS/Native bridge is also a big constraint: single-threaded, JSON transports (bytes are base64 encoded), and it shares no memory references so payloads get copied around and processed one-at-a-time. As a result, sending lots of data back/forth is a reliable way to slow an app to a crawl.
• We tried doing this three different ways:
  • swapping out just-the-transport layer for gRPC and relaying req/res payloads, and
  • relaying just encoded-content back and forth.
  • Both of those attempts overwhelmed the bridge. So we also tried using the "New Architecture" which is an experimental RN system that lets you expose native code via JSI. This avoided the bridge for gRPC req/res but it complicated the build and was still unusably slow because the JS engine itself was still slow at doing the crypto/byte-data work. We stopped short of rewriting xmtp-js + all JS codecs to have a parallel edition that used JSI bindings. Instead we opted for this architecture where all heavy lifting is in native code and payloads across the bridge are kept small.

Performance was the driving reason but I do think there's also an API/SDK design rationale that holds here too: react native is not a web context and developers are mislead trying to treat it that way. Native apps really do have fundamentally different architectures and expectations from web apps. The browser does a lot for you and your webpage doesn't run in your pocket. And IMO xmtp-js is actually better understood as xmtp-web-sdk -- it works great for web apps. But native app SDKs should feel different and be designed to support native UX flows (push notifs, auth, files, media playback, etc). Long term, this is why I think the RN SDK should be relatively easy to maintain as a wrapper on the iOS/Android SDKs (because they should have the same API).

[nmalzieu]

Totally agree! However I guess for "simple content types" that are basically some metadata, being able to code them in JS then use them in RN would be nice - and that would probably work as @neekolas said: just ship bytes out and do the work in JS. This might be subperformant in some cases but would make all content types available in RN which would also have great value - I guess that's exactly the goal of this task so we're on the right path!

[nplasterer]

Implemented in https://github.com/xmtp/xmtp-react-native/pull/155