search

F33RNI / LlM-Api-Open

Unofficial open APIs for popular LLMs (currently for ChatGPT and MS Copilot) with self-hosted redirect capability
MIT License
29 stars 5 forks source link
api bing-ai browser-automation chatgpt chrome copilot hacked microsoft open-source proxy proxy-server reverse-engineering selenium telegram unofficial

readme

🕊️ LLM-API-Open (LMAO)

LLM-API-Open logo

Unofficial open APIs for popular LLMs with self-hosted redirect capability

❓ WAT

🕊️ LLM-API-Open (LMAO) allows for the free and universal utilization of popular Large Language Models (LLM). This is achieved using browser automation. LLM-API-Open (LMAO) launches a browser in headless mode and controls a website as if a real user were using it. This enables the use of popular LLMs that usually don't offer easy and free access to their official APIs

🔥 Additionally, LLM-API-Open (LMAO) is capable of creating its own API server to which any other apps can send requests. In other words, you can utilize LLM-API-Open (LMAO) both as a Python package and as an API proxy for any of your apps!

🚧 LLM-API-Open is development

Due to my studies, I don't have much time to work on the project 😔

Currently, LLM-API-Open has only 2 modules: ChatGPT and Microsoft Copilot

📈 But it is possible to add other popular online LLMs (You can wait, or make a pull-request yourself)

📄 Documentation is also under development! Consider reading docstring for now

😋 Support project

🏗️ Getting started

⚠️ Will not work with Python 3.13 or later due to imghdr

⚙️ 1. Download / build / install LLM-API-Open

There is 4 general ways to get LLM-API-Open

⚙️ Install via pip

⬇️ Download cli version from releases

https://github.com/F33RNI/LLM-API-Open/releases/latest

🔨 Build cli version from source using PyInstaller

git clone https://github.com/F33RNI/LLM-API-Open.git
cd LLM-API-Open

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

pyinstaller lmao.spec

dist/lmao --help

💻 Use source as is

git clone https://github.com/F33RNI/LLM-API-Open.git
cd LLM-API-Open

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

export PYTHONPATH=./src:$PYTHONPATH
export PYTHONPATH=./src/lmao:$PYTHONPATH
python -m main --help

🔧 2. Configure LLM-API-Open

  1. Download configs directory from this repo
  2. Open .json files of modules you need in any editor and change it as you need
  3. Specify path to configs directory with -c path/to/configs argument

📦 Python package example

import logging
import json

from lmao.module_wrapper import ModuleWrapper

# Initialize logging in a simplest way
logging.basicConfig(level=logging.INFO)

# Load config
with open("path/to/configs/chatgpt.json", "r", encoding="utf-8") as file:
    module_config = json.loads(file.read())

# Initialize module
module = ModuleWrapper("chatgpt", module_config)
module.initialize(blocking=True)

# Ask smth
conversation_id = None
for response in module.ask({"prompt": "Hi! Who are you?", "convert_to_markdown": True}):
    conversation_id = response.get("conversation_id")
    response_text = response.get("response")
    print(response_text, end="\n\n")

# Delete conversation
module.delete_conversation({"conversation_id": conversation_id})

# Close (unload) module
module.close(blocking=True)

💻 CLI example

$ lmao --help        
usage: lmao [-h] [-v] [-c CONFIGS] [-t TEST] [-i IP] [-p PORT] [-s SSL [SSL ...]] [--tokens-use TOKENS_USE [TOKENS_USE ...]]
            [--tokens-manage TOKENS_MANAGE [TOKENS_MANAGE ...]] [--rate-limits-default RATE_LIMITS_DEFAULT [RATE_LIMITS_DEFAULT ...]] [--rate-limit-fast RATE_LIMIT_FAST]
            [--no-logging-init]

Unofficial open APIs for popular LLMs with self-hosted redirect capability

options:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  -c CONFIGS, --configs CONFIGS
                        path to configs directory with each module config file (Default: configs)
  -t TEST, --test TEST  module name to test in cli instead of starting API server (eg. --test=chatgpt)
  -i IP, --ip IP        API server Host (IP) (Default: localhost)
  -p PORT, --port PORT  API server port (Default: 1312)
  -s SSL [SSL ...], --ssl SSL [SSL ...]
                        Paths to SSL certificate and private key (ex. --ssl "path/to/certificate.crt" "path/to/private.key")
  --tokens-use TOKENS_USE [TOKENS_USE ...]
                        API tokens to enable authorization for /status, /ask, /stop and /delete (ex. --tokens-use "tokenForMyApp" "tokenForMyAnotherApp"
                        "ultraPrivateTokeeeeeen")
  --tokens-manage TOKENS_MANAGE [TOKENS_MANAGE ...]
                        API tokens to enable authorization for /init and /close (ex. --tokens-manage "ultraPrivateTokeeeeeen")
  --rate-limits-default RATE_LIMITS_DEFAULT [RATE_LIMITS_DEFAULT ...]
                        Rate limits for all API requests except /stop (Default: --rate-limits-default "10/minute", "1/second")
  --rate-limit-fast RATE_LIMIT_FAST
                        Rate limit /stop API request (Default: "1/second")
  --no-logging-init     specify to bypass logging initialization (will be set automatically when using --test)

examples:
  lmao --test=chatgpt
  lmao --ip="0.0.0.0" --port=1312
  lmao --ip="0.0.0.0" --port=1312 --no-logging-init
  lmao --ip "0.0.0.0" --port=1312 --ssl certificate.crt private.key --tokens-use "tokenForMyApp" "tokenForMyAnotherApp" "ultraPrivateTokeeeeeen" --tokens-manage "ultraPrivateTokeeeeeen"
$ lmao --test=chatgpt
WARNING:root:Error adding cookie oai-did
WARNING:root:Error adding cookie ajs_anonymous_id
WARNING:root:Error adding cookie oai-allow-ne
User > Hi!    
chatgpt > Hello! How can I assist you today?

🌐 API example

Start server

Please see 🔒 HTTPS server and token-based authorization section for more info about HTTPS server and tokens

$ lmao --configs "configs" --ip "0.0.0.0" --port "1312" 
2024-03-30 23:14:50 INFO     Logging setup is complete
2024-03-30 23:14:50 INFO     Loading config files from configs directory
2024-03-30 23:14:50 INFO     Adding config of ms_copilot module
2024-03-30 23:14:50 INFO     Adding config of chatgpt module
 * Serving Flask app 'lmao.external_api'
 * Debug mode: off
2024-03-30 23:14:50 INFO     WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:1312
 * Running on http://192.168.0.3:1312
2024-03-30 23:14:50 INFO     Press CTRL+C to quit
...

🐍 Python (requests)

import logging
import time
from typing import Dict

import requests

# API URL
BASE_URL = "http://localhost:1312/api"

# Timeout for each request
TIMEOUT = 60

# Initialize logging in a simplest way
logging.basicConfig(level=logging.INFO)

def post(endpoint: str, data: Dict):
    """POST request wrapper"""
    response_ = requests.post(f"{BASE_URL}/{endpoint}", json=data, timeout=TIMEOUT, stream=endpoint == "ask")
    if endpoint != "ask":
        try:
            logging.info(f"{endpoint.capitalize()} Response: {response_.status_code}. Data: {response_.json()}")
        except Exception:
            logging.info(f"{endpoint.capitalize()} Response: {response_.status_code}")
    else:
        logging.info(f"{endpoint.capitalize()} Response: {response_.status_code}")
    return response_

def get(endpoint: str):
    """GET request wrapper"""
    response_ = requests.get(f"{BASE_URL}/{endpoint}", timeout=TIMEOUT)
    logging.info(f"{endpoint.capitalize()} Response: {response_.status_code}. Data: {response_.json()}")
    return response_

# Initialize module
response = post("init", {"module": "chatgpt"})

# Read module's status and wait until it's initialized (in Idle)
logging.info("Waiting for module initialization")
while True:
    response = get("status")
    chatgpt_status_code = response.json()[0].get("status_code")
    if chatgpt_status_code == 2:
        break
    time.sleep(1)

# Ask and read stream response
response = post("ask", {"chatgpt": {"prompt": "Hi! Please write a long text about AI", "convert_to_markdown": True}})
logging.info("Stream Response:")
for line in response.iter_lines():
    if line:
        logging.info(line.decode("utf-8"))

# Delete last conversation
response = post("delete", {"chatgpt": {"conversation_id": ""}})

# Close module (uninitialize it)
response = post("close", {"module": "chatgpt"})

🌐 CURL

For CURL examples please read 📄 API docs section

📄 API docs

⚠️ Documentation is still under development!

🌐 Module initialization /api/init

Begins module initialization (in a separate, non-blocking thread)

Please call /api/status to check if the module is initialized BEFORE calling /api/init.

After calling /api/init, please call /api/status to check if the module's initialization finished.

Request (POST):

Maximum content length: 100 bytes. Default rate limits: 10/minute, 1/second

Returns:

Example:

$ curl --request POST --header "Content-Type: application/json" --data '{"module": "chatgpt"}' http://localhost:1312/api/init
{}

🌐 Status /api/status

Retrieves the current status of all modules

Request (POST):

Maximum content length: 100 bytes. No rate limit

Returns:

[
    {
        "module": "Name of the module from MODULES",
        "status_code": "Module's status code as an integer",
        "status_name": "Module's status as a string",
        "error": "Empty or module's error message"
    },
]

Example:

$ curl --request POST --header "Content-Type: application/json" --data '{}' http://localhost:1312/api/status
[{"error":"","module":"chatgpt","status_code":2,"status_name":"Idle"}]

🌐 Send request and get stream response /api/ask

Initiates a request to the specified module and streams responses back (if no_stream is false or not specified)

Please call /api/status to check if the module is initialized and not busy BEFORE calling /api/ask

To stop the stream / response, please call /api/stop

Request (POST):

Maximum content length: 100 bytes. Default rate limits: 10/minute, 1/second

Yields (or Returns if "no_stream" is specified):

For ChatGPT, each JSON object has the following structure:

{
    "finished": "True if it's the last response, False if not",
    "message_id": "ID of the current message (from assistant)",
    "response": "Actual response as text"
}

For Microsoft Copilot, each JSON object has the following structure:

{
    "finished": True if it's the last response, False if not,
    "response": "response as text (or meta response)",
    "images": ["array of image URL's"],
    "caption": "images caption",
    "attributions": [
        {
            "name": "name of attribution",
            "url": "URL of attribution"
        },
        ...
    ],
    "suggestions": ["array of suggestions of the requests"]
}

Returns:

Example:

$ curl --request POST --header "Content-Type: application/json" --data '{"chatgpt": {"prompt": "Hi! Who are you?", "convert_to_markdown": true}}' http://localhost:1312/api/ask
{"finished": false, "conversation_id": "1033be5b-d37d-46b3-b47c-9548da5b192c", "message_id": "00d9cc0d-c4d9-484d-a8e5-9c78eaf2a0e1", "response": "Hello! I'm ChatGPT, an AI developed by O"}
...
{"finished": true, "conversation_id": "1033be5b-d37d-46b3-b47c-9548da5b192c", "message_id": "00d9cc0d-c4d9-484d-a8e5-9c78eaf2a0e1", "response": "Hello! I'm ChatGPT, an AI developed by OpenAI. I'm here to help answer your questions, engage in conversation, provide information, or assist you with anything else you might need. How can I assist you today?"}

🌐 Stop stream response /api/stop

Stops the specified module's streaming response (stops yielding from /api/ask)

Request (POST):

Maximum content length: 100 bytes. Default rate limits: 1/second

Returns:

Example:

$ curl --request POST --header "Content-Type: application/json" --data '{"module": "chatgpt"}' http://localhost:1312/api/stop
{}

🌐 Delete conversation /api/delete

Clears the module's conversation history

Please call /api/status to check if the module is initialized and not busy BEFORE calling /api/delete

Request (POST):

Maximum content length: 500 bytes. Default rate limits: 10/minute, 1/second

Returns:

Example:

$ curl --request POST --header "Content-Type: application/json" --data '{"chatgpt": {"conversation_id": "1033be5b-d37d-46b3-b47c-9548da5b192c"}}' http://localhost:1312/api/delete
{}

🌐 Close module /api/close

Requests the module's session to close (in a separate, non-blocking thread)

Please call /api/status to check if the module is initialized and its status is Idle or Failed BEFORE calling /api/close

After calling /api/close, please call /api/status to check if the module's closing finished

Request (POST):

Maximum content length: 500 bytes. Default rate limits: 10/minute, 1/second

Returns:

🔒 HTTPS server and token-based authorization

⚠️ Better use proper SSL service and redirect to local port

⚠️ Don't use token-based authorization with bare HTTP (without any SSL). It's not safe!

It's possible to start SSL (HTTPS) server instead of HTTP. For that, provide --ssl argument with path to certificate file and path to private key file.

Example:

$ lmao --configs "configs" --ip "0.0.0.0" --port "1312" --ssl certificate.crt private.key
2024-04-19 02:09:10 INFO     Logging setup is complete
2024-04-19 02:09:10 INFO     Loading config files from configs directory
2024-04-19 02:09:10 INFO     Adding config of ms_copilot module
2024-04-19 02:09:10 INFO     Adding config of chatgpt module
2024-04-19 02:09:10 WARNING  No tokens provided. Everyone can use API
2024-04-19 02:09:10 INFO     Rate limits for all API requests except /status and /stop: 10/minute, 1/second
2024-04-19 02:09:10 INFO     Rate limits /status and /stop API requests: 1/second
 * Serving Flask app 'lmao.external_api'
 * Debug mode: off
2024-04-19 02:09:10 INFO     WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on https://127.0.0.1:1312
 * Running on https://192.168.0.3:1312
2024-04-19 02:09:10 INFO     Press CTRL+C to quit
...

Also you can enable token-based authorization. For that, provide --tokens-use argument with a list of some strong random tokens. Requests that provided these tokens can access /status, /ask, /stop and /delete. For /init and /stop, provide --tokens-manage argument with a list of some REALLY strong random tokens

⚠️ Make sure you provided at least one token to --tokens-manage if you're using --tokens-use. Otherwise EVERYONE CAN ACCESS /init and /stop

Example (tokens for /status, /ask, /stop and /delete: naixae3eeNao6suu, kahMeixoo9un9OhR. Token for /init and /stop: ofi2ohRi8maish4x):

$ lmao --configs "configs" --ip "0.0.0.0" --port "1312" --ssl certificate.crt private.key --tokens-use naixae3eeNao6suu kahMeixoo9un9OhR --tokens-manage ofi2ohRi8maish4x
2024-04-19 02:07:02 INFO     Logging setup is complete
2024-04-19 02:07:02 INFO     Loading config files from configs directory
2024-04-19 02:07:02 INFO     Adding config of ms_copilot module
2024-04-19 02:07:02 INFO     Adding config of chatgpt module
2024-04-19 02:07:02 INFO     Token-based authorization enabled. Provided 2 tokens-use
2024-04-19 02:07:02 INFO     Token-based authorization enabled. Provided 1 tokens-manage
2024-04-19 02:07:02 INFO     Rate limits for all API requests except /status and /stop: 10/minute, 1/second
2024-04-19 02:07:02 INFO     Rate limits /status and /stop API requests: 1/second
 * Serving Flask app 'lmao.external_api'
 * Debug mode: off
2024-04-19 02:07:02 INFO     WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:1312
 * Running on http://192.168.0.3:1312
2024-04-19 02:07:02 INFO     Press CTRL+C to quit
...

🐛 How to fix Failed to establish a new connection when using multiple modules

  1. Open venv/lib/python3.XX/site-packages/undetected_chromedriver/patcher.py
  2. Add import secrets to the imports
  3. In def __init__ change prefix = "undetected" to prefix = f"undetected{secrets.token_hex(4)}"
  4. Save file and reload LMAO