Moving to Expo and other ways to improve DX

[gmaclennan]

Developer Experience (DX) for Mapeo Mobile has not been great. Some key challenges include:

1. Setting up the Android build environment (Android Studio, SDK, Gradle) is a challenge for any contributor.
2. Compounding (1), our backend code in nodejs-mobile includes native C++ code, which needs to be compiled from source using NDK. Setting up NDK and configuring npm & node-gyp for the build environment is an additional challenge on top of the Android Studio setup.
3. Updating React Native involves complex updates to all the files in the android and ios folders, and keeping that in track with our own modifications to these folders is major challenge.
4. We often have challenges with native (Java) modules not working with different versions of React Native.
5. Our use of v8 (rather than JSC) for a JS engine creates additional headaches when upgrading. This change was initially made for better performance and a lower memory footprint, but Hermes might be the better supported option going forwards.
6. Building the app on Bitrise is a constant challenge, because the machine setup changes on Bitrise's end, and new versions of React Native require different SDK tools to be installed.

One route towards improving the developer experience is to make more use of the Expo ecosystem. We already use some Expo modules in Mapeo Mobile — they are often the best maintained react native modules and are quickly updated to support new versions of RN. There are two ways we can leverage Expo, with different resource requirements:

1. Expo "bare" project

Expo can work with an existing "bare" React Native app. We can use Expo Application Services (Expo's PaaS for building and publishing apps), and we can use the expo modules & cli to facilitate local development. However we still need to manage the contents of android and ios folders ourselves. With a bare project we can create prebuilds, which allow developers to develop the front end and view on a device/emulator without needing the Android or iOS build setup: the frontend JS code can load in the prebuild, which can be built by one person / CI every time we change a native module, and then developers can install that prebuild and develop Mapeo in that.

2. Expo fully managed project

If you start an expo project from scratch, Expo fully manages the "native" side of the app, and the android and ios files can be generated by expo at build-time. As long as we do not edit those folders then we don't need to check them in, and we don't need to worry about them for updates. Expo supports native modules that are not included in the Expo SDK via plugins. Plugins are a config that tells expo how to modify the generated native code. For example Mapbox-gl maps includes config for installing into an expo project.

The advantage of (2) is that it should simplify the process of updating and maintaining the app - we should never need to touch any gradle config or android or ios config, since Expo will generate all of that for us. There are some key development challenges for (2) though:

1. We will need to create an Expo plugin configuration for nodejs-mobile-react-native.
2. We need to figure out the relevant expo config needed for things like our debug builds that depend on changes we have made to the native side of the app. I think most things we need are supported in Expo though, the only exception being the config to build the Mapeo for ICCAs app from the same codebase, but we do not plan to do that with Mapeo 6, so we can drop it.

Improvements beyond Expo

One of the biggest challenges is configuring the build environment for native C++ modules that we use on the nodejs-mobile side. Configuring NDK correctly has consistently proved a major headache. nodejs-mobile however does support prebuilds, which remove the need to do any building from source at build time. Solution: We setup some CI processes to automatically build a publish prebuilds for the native node modules that we use, e.g. better-sqlite3 and sodium-native. We publish these with a separate name, but will be forks of the original modules e.g. @mapeo/better-sqlite3. We can automate updating these when the original module updates. This means we can set up the build environment once in CI, and for developers they only need to download these prebuidls.

Another option we can look at for a nice DX is to conditionally require non-expo native modules so that developers can use the Expo Go client: https://docs.expo.dev/bare/using-expo-client/. This would mean having a placeholder for the map screen and mocking the Mapeo API, because it would not have the nodejs process. However the advantage would be that you can view Mapeo in the browser, or very easily start developing on the screens (other than the Map screen) with just the regular Expo Go client.

[achou11]

Closing, as we've committed to using option (2)