LocalGPT-Android

• The Local GPT Android is a mobile application that runs the GPT (Generative Pre-trained Transformer) model directly on your Android device. This app does not require an active internet connection, as it executes the GPT model locally.
• (It still uses Internet to download the model, you can manually place the model in data directory and disable internet).

Requirements

• Android 11+
• At least 3-4 GB of Free RAM (Again "Free" RAM)
• I tested it on iQoo 11 with 16GB RAM and Snapdragon 8 Gen 2

Features

• It works :)
• Only Text Completions as of now
  • If you want to to chat, send messages like ### Human:\n prompt\n ### Assistant:
• The model used does not care about ethics afaik

How to?

• Download the latest release
• Wait for the download to complete
  • The model is ggml-gpt4all-j-v1.3-groovy.bin (Downloaded from gpt4all.io)
• The model will get loaded
• You can start chatting

Benchmarks

Demo

• Demo Video or https://youtube.com/shorts/J7DJ-40Uy1k?feature=share

TODO

• [x] Add very basic chat support
• [ ] Reduce the size of the model
• [ ] Fix UI issues

Notes

• It is very much buggy. I haven't tested it much.
• You'll need a good device to run this.
• Response initiation time and RAM usage for Chat Completion increases with the number of messages.
  • This is because Chat Completion is using Text Completion, and with every message the prompt size increases.
  • This could be fixed by training the model with Chat model in mind. I'm looking into this.

Credits

• nomic-AI