search

kambo-1st / llama-cpp-php

The package enables the use of the LLama C++ library in PHP, thereby allowing the setup and execution of LLM models in PHP on your local machine.
MIT License
41 stars 5 forks source link

readme

Experimental bindings for LLama C++ library

Latest Version on Packagist Tests Total Downloads

The package enables the use of the LLama C++ library in PHP, thereby allowing the setup and execution of LLM models in PHP on your local machine.

This is highly experimental and not suitable for production use!

Use at your own risk!

Only Linux is supported!

asciicast

Installation

You can install the package via composer:

composer require kambo/llama-cpp-php kambo/llama-cpp-php-linux-lib

Note: the kambo/llama-cpp-php-linux-lib package contains a binary library for Linux.

Usage

Get model, you can use for example this command:

wget https://huggingface.co/LLukas22/gpt4all-lora-quantized-ggjt/resolve/main/ggjt-model.bin
$template = "You are a programmer, write PHP class that will add two numbers and print the result. Stop at class end.";
$context = Context::createWithParameter(new ModelParameters(__DIR__ .'/models/ggjt-model.bin'));
$llama = new LLamaCPP($context);
echo "Prompt: \033[0;32m".$template."\033[0m".PHP_EOL;

foreach ($llama->generate($template, new GenerationParameters(predictLength: 200)) as $token) {
    echo $token;
}

License

The MIT License (MIT). Please see License File for more information.