louis-jan
commented
1 month ago Jan on Mobile architecture
Background
Jan's mobile app was first built on React Native, which brought our first inference UI to the world. We maintained it for a while, but the problem is that it's scoped at mobile UI only and not really designed to be scaled to other platforms such as Desktop.
We built Jan Desktop, an Electron application with extensibility. Let's take a deep look at how we built it.
| Extensible Jan Architecture |
|---|
Electron is good, but not for mobile
Electron is built on two major upstream projects: Chromium and Node.js. Neither of those is mobile-friendly, especially within the Apple ecosystem.
To make Electron work on mobile, we have to deal with both tough challenges simultaneously. I stopped looking for a solution that sticks to Electron. I don’t want to join a two-on-one fight.
Capacitor is a popular candidate
I have read about various projects, and they have chosen Capacitor to port their apps to mobile, including Obsidian. There have been a lot of discussions about this, and all the conclusions seem positive. I tried to initialize a Capacitor about an hour ago, and sadly I realized that… Capacitor can only run web-compliant code, not Node.js code.
It can just run, but it's not scalable. We can port our extensive code base to run on the web and native code, but that's a lot of work. And it's really platform-specific, which isn't what we're looking for.
| Capacitor Structure |
|---|
| Capacitor is good for driving stuff behind a web app only |
Let's take a deeper look at how capacitors work. It gets webDir as the main entry. Which mean it’s very web app driven specific. Thus, it creates a bridge somewhere under the hood, between JS and Native APIs, which is a great approach, one that React Native and others are also taking.
import { CapacitorConfig } from '@capacitor/cli';
const config: CapacitorConfig = {
appId: 'com.company.appname',
appName: 'My Capacitor App',
webDir: 'www',
};
export default config;However, a browser's JS runtime is not the solution we are looking for.
There is no alternative to Electron on Mobile?
I tried searching the intranet for an alternative to Electron that can run a NodeJS backend on mobile and communicate with a native browser process but with no luck.
NodeJS is not mobile-friendly!!!
There is still a great candidate left
Tauri - the one we love, but it has not joined our game yet.
How cool is it that Tauri recently announced beta support for mobile. I knew it would happen, but I didn’t expect it to be that good.
- Tauri is designed to be pretty close to Electron. It uses Rust at its core backend and a native webview for the renderer process.
- Tauri supports native plugins!!! WOW. We can build Tauri plugins with native modules using Kotlin, or Swift. WUT?
Those two are very important because:
First, we designed our architecture to be API-friendly, replaceable, and extensible.
- Native API friendly: we build on top of a high-performance inference core (inference engines). It would be very hard to embed these in the browser's JS runtime, which is actually not possible, except in a tricky way.
- Replaceable: We built Jan Backend as an abstraction layer, so the renderer process does not know which backend is plugged in. We are not limited by language, framework or platform.
- Extensible: We built the Jan web app to be very extensible, with all the logic provided by extensions. We can replace any extension that does not perform well on any design, architecture, or platform.
Rust has a very minimal, no-setup-required runtime and can be run anywhere.
Second, mobile is a different game; it's very native-specific. Using a native webview and modules is a great idea.
Tauri supports plugins: Plugins are reusable extensions to the Tauri API that solve common problems.
It also supports mobile plugins!
Plugins can run native mobile code written in Kotlin (or Java) and Swift. The default plugin template includes an Android library project using Kotlin and a Swift package including an example mobile command showing how to trigger its execution from Rust code.
I tested in an hour and got into it.
| Tauri with native mobile plugin |
|---|
| Tauri Mobile Plugin - With Native Package Manager Support - Native Frameworks and Language Support |
I felt like I was developing a mini iOS app inside Jan.
Back to our architecture to see how it fits
The same as Electron, which uses NodeJS. We replace the backend handlers for exposed APIs with frontend ones (e.g. FileSystem). But there is one more layer added to grant us full control over the platform-specific frameworks: Native Plugin.
| Jan Architecture |
|---|
| Only one extra layer is needed for full Native API control |
Another look at the code structure
| Modules Structure |
|---|
Test results
This is just a test on a small Tauri project on Mobile to test the possibility of:
- Reusing our entire webapp code base (Jan NextJS App)
- Communication between components, from web app to Rust backend, to native mobile API
- Embed an inference engine, such as Llama.cpp.
Let’s goooo!!!
Try to run a Next JS app on Mobile
I actually tested it on desktop a couple of months ago. Now it’s time to run it on mobile.
Build, plugin and communicate with a Native Plugin
One plugin can provide three modules that work with one interface. The native API can be invoked from Rust or Webapp.
Embed an inference engine
As the screenshot above, we can add llama.cpp swift package and use it
This file is from https://github.com/ggerganov/llama.cpp
The example plugin:
You can invoke the plugin functions directly from the web app.
The greet function (row 11 → 15) is an example of:
- Invoke a backend function and get result
- Invoke a native plugin function and get result
There is more
Updating...
dan-homebrew
ilsubyeega
Goal
Resources
Goal is for Jan's architecture to be scalable from Mobile to Desktop to Server