Bridging layer research for Mapeo-Core-Next

[ErikSin]

Research different architectures for bridging layers for Mapeo-Core-Next. Document the pros and con's of each, and that will be used to inform how we integrate Mapeo-Core-Next. We will eventually prototype the different types of architecture (For each architecture protyped please create a new ticket).

To Do:

• [ ] Document (in this ticket) High level architecture of current bridging layer. Try to include the reasons when architectural decisions were made
• [ ] Research the best/most common practices for bridging layer and document (in this ticket) reason we may not be able to follow best practices
• [ ] Determine what criteria we are using for choosing architecture
• [x] Create repo using mapeo-core-next, node-js-mobile, and react-native-frontend
• [ ] Create tickets detailing what types of architectures we are prototyping

[achou11]

met today to discuss high level architecture. points of discussion included:

• how we build the backend code that nodejs mobile eventually loads
• why we include certain files in the backend (e.g. loader.js, workers)
• the general steps and lifecycle of the gradle tasks that nodejs-mobile-react-native performs to move files around and build native code
• documentation for rn-bridge apis: https://code.janeasystems.com/nodejs-mobile/react-native/bridge
• prototyping goals:
  • how does rpc-reflector get incorporated in this context?

[tomasciccola]

I've made rpc-reflector work on the starter project. Some caveats to take into account:

• assert, buffer and readable-stream need to be installed on the root project (they're dependencies of rpc-reflector, do they actually need to be explicitly installed?)
• on metro.config.js we need to add

  extraNodeModules: {
    stream: require.resolve('readable-stream')
  }

  inside the resolver field, so that readable-stream can be required as stream

• there's two wrappers to the react native channels (front and back end) that live on src/{backend,frontend}/lib/message-port-like.js. For some reason rn_bridge.channel (backend) and nodejs.channel (frontend) work different, so this file can't be shared.
• for now, rpc-reflector is patched on a couple of files (probably soon those can be merge into the library)
• every rpc call is async, despite the call being sync, so to get a useful value we have to await the call always. Since the api is typed, we would get a warning with 'await' has no effect on the type of expression
• There's a type error (see here) where ts complains that the setter of the useState expects a SetStateAction<typeof api> instead of ClientApi<typeof api>. I tried to type it as <SetStateAction<typeof api>> but that didn't solve the error. Despite that the created api object on the client since to be correctly typed by the editor tooling

One thing that I would like to try is having events sent from the backend without any rpc call from the client. I don't know if rpc-reflector provides a way to do this or if we should directly use nodejs.channel.addListener and rn_bridge.channel.send but probably @gmaclennan knows the answer to this. I'm closing this for know (but it can be reopen if anybody feels there's more research to be done)

[gmaclennan]

One thing that I would like to try is having events sent from the backend without any rpc call from the client.

@tomasciccola could you expand on this a bit more? What use case are you imagining? rpc-reflector does support event emitters and subscribing to events, so that on the server you can do:

class MyApi extends EventEmitter {
  constructor() {
    super()
    setInterval(() => this.emit('ticktock'), 1000)
  }
}

and on the client you can do:

const myApi = createClient(channel)
myApi.on('ticktock', () => console.log('ticktock'))

At least, that is how it should work.

If you need other events for coordinating server startup etc, then I recommend using a separate channel of communication, e.g. outside of the rpc-reflector instance. In nodejs mobile you can do this by using different event names, e.g. the default is message where-as for the example here you use @@API_MESSAGE.

[tomasciccola]

the use case I'm imagining is smth like the server sending update events that don't span from a call from the client. I don't know if applies to how mapeo-core is designed; but imagine smth like

// server
this.emit('observation:update', {id, ...})

where a record is updated and we want to send the updated version to the client without the client explicitly calling for it (like a live update)... Again, I don't know if that's a use case we have; it was just a doubt that I had...