Mobile App version of AnythingLLM?

[arncore]

What would you like to see?

Not sure this is the right place to ask for this but is there any desire to have mobile apps for anythingllm? Any work in progress in that regard?

If not, I would love to explore the possibility of building one. I run AnythingLLM on my server at home. Would be nice to have a mobile app to interface with it rather than relying just on the desktop app.

[timothycarambat]

We do not have any plans to build out a mobile app currently. This we feel is handled by the Docker browser mobile UI (which does need improvement).

The current reasoning is:

• Limited ability to support for iOS and Android for core-team
• Lack of ability to do many on-device functions, relying on the need to connect to some kind of server based API to support the frontend (we are now back to just using it in browser).
  • If you wanted to connect a mobile client to a desktop instance you would need to tunnel the connection to be publically accessible. This is inherently not secure and we dont want to encourage this behavior
• Another thing to manage, which is already limited currently between Docker/Desktop

[arncore]

Hey, thanks for your reply. Appreciate the thoughts on this.

1. I understand the resource restraint issue. I am an Android lead at Capital One. This question arose from my desire to possibly contribute to this project given I align with its goals. I would have no problem driving this effort myself.

2. I am bit unclear on this point here. Not sure I'm following regarding the on device functions. The assumption here is that the LLM runs on dedicated hardware, through an LLM provider or machine in local network, and never on the phone. Can some LLMs run on phones? Sure, but thats not my intended use for the mobile app. Its more of a companion app to the anythingllm web app but with document or photo support. Perhaps in the future it could facilitate running mobile LLMs if circumstances allow.

Yes, there needs to be an API that the mobile app calls to interact with the LLM. Im assuming the desktop app is doing this right now, isnt it?

I havent looked but I guess if you package the anythingllm web app with an electron runtime and file reading then you kinda have yourself a desktop app without the need to make an API. I think this is what you're describing.

Assuming the latter, is there plans to create an API? I feel like this would be the natural progression of the project. And to be honest, just thinking about this, it's no easy feat creating a common interface that applies to each model. Some dont have support for photos, some do etc. Very interested to hear about this.

[timothycarambat]

I believe we are on the same page, I think I was actually just talking ahead of myself in my previous comment.

So for point 2 this was my thought process:

• Mobile app communicates with a running AnythingLLM instance
• So AnythingLLM needs and API to support a dedicated mobile app
• Its currently easier to support the mobile web client, albeit not native so plenty of downsides.
• Of course, since your cant run an express server in a mobile app. The mobile app is a native interface for AnythingLLM running somewhere. As one would expect since that's how most mobile apps work.

Making a native UI mobile client API for AnythingLLM can be done very reasonably! The full API runs automatically on all forms of AnythingLLM (Docker and Desktop) - so it for sure can exist. The main hangup is that when a person is running the desktop app everything is in that desktop app. There is no cloud server, so how does one securely expose their client machine running the desktop app to the "public" so the app can connect to it?

Expo kind of does this already, but they centralize the tunnel between the desktop and mobile client. That is really the only detail that needs to be ironed out for AnythingLLM Mobile to exist

[arncore]

Yup, I think we're on the same page now 😄

One way to do the connection is by having the docker instance exposed behind a proxy server, behind cloudflare. Data would be encrypted in transit (TLS cert) and on top of that data is already encrypted on device. For example, the mobile app can import the RSA public key and use it to encrypt the requests. Not sure how you guys are encrypting things but there's different options here.

Exposing the desktop instance is definitely trickier considering it may run on a machine that may even turn off from time to time. However, if people wanna expose the desktop app they can as well.

This is all under the assumption that anythingllm can expose a server for clients (other instances of itself in this case) to connect to, if desired. Considering anythingllm was designed to be offline first Im assuming this doesn't exist. This means that one of the instances needs to act as a "primary" and the rest are "secondaries". On a secondary instance the vector db and the other unnecessary pieces can be disabled since the primary is storing things and its acting as a single source of truth. All the calls to the secondary's internal API are instead made on the exposed API of the primary.

All this does make anythingllm a monolith lol. So that's something to consider. But under resource and time constraints thats the best option. Eventually, the API/server can be decoupled from the clients as needed and it would be a major refactor.