Use tool with ollama

[devbysb]

Hi,

Is it possible to use tools with ollama ?

I have found this article https://ollama.com/blog/tool-support

I have tried with OllamaChat() but it's not implemented.

I also tried with OpenAiChat() but I can specify the url of ollama server (to test with openAi compatibility)

Thank you

[MaximeThoonsen]

Yes we should update the class so we can use tools with Ollama

[Gudetomi]

I have with this problem to use Ollama

"message": "cURL error 7: Failed to connect to localhost port 11434 after 0 ms: Couldn't connect to server (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://localhost:11434/api/embeddings", "exception": "GuzzleHttp\Exception\ConnectException", "file": "/var/www/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php", "line": 275, "trace": [ { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php", "line": 205, "function": "createRejection", "class": "GuzzleHttp\Handler\CurlFactory", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php", "line": 157, "function": "finishError", "class": "GuzzleHttp\Handler\CurlFactory", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Handler/CurlHandler.php", "line": 47, "function": "finish", "class": "GuzzleHttp\Handler\CurlFactory", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Handler/Proxy.php", "line": 28, "function": "invoke", "class": "GuzzleHttp\Handler\CurlHandler", "type": "->" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Handler/Proxy.php", "line": 48, "function": "GuzzleHttp\Handler\{closure}", "class": "GuzzleHttp\Handler\Proxy", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php", "line": 64, "function": "GuzzleHttp\Handler\{closure}", "class": "GuzzleHttp\Handler\Proxy", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Middleware.php", "line": 31, "function": "invoke", "class": "GuzzleHttp\PrepareBodyMiddleware", "type": "->" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/RedirectMiddleware.php", "line": 71, "function": "GuzzleHttp\{closure}", "class": "GuzzleHttp\Middleware", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Middleware.php", "line": 66, "function": "invoke", "class": "GuzzleHttp\RedirectMiddleware", "type": "->" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/HandlerStack.php", "line": 75, "function": "GuzzleHttp\{closure}", "class": "GuzzleHttp\Middleware", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Client.php", "line": 333, "function": "invoke", "class": "GuzzleHttp\HandlerStack", "type": "->" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Client.php", "line": 169, "function": "transfer", "class": "GuzzleHttp\Client", "type": "->" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Client.php", "line": 189, "function": "requestAsync", "class": "GuzzleHttp\Client", "type": "->" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/ClientTrait.php", "line": 95, "function": "request", "class": "GuzzleHttp\Client", "type": "->" }, { "file": "/var/www/vendor/theodo-group/llphant/src/Embeddings/EmbeddingGenerator/Ollama/OllamaEmbeddingGenerator.php", "line": 38, "function": "post", "class": "GuzzleHttp\Client", "type": "->" }, { "file": "/var/www/vendor/theodo-group/llphant/src/Query/SemanticSearch/QuestionAnswering.php", "line": 78, "function": "embedText", "class": "LLPhant\Embeddings\EmbeddingGenerator\Ollama\OllamaEmbeddingGenerator", "type": "->" }, { "file": "/var/www/vendor/theodo-group/llphant/src/Query/SemanticSearch/QuestionAnswering.php", "line": 28, "function": "searchDocumentAndCreateSystemMessage", "class": "LLPhant\Query\SemanticSearch\QuestionAnswering", "type": "->" }, { "file": "/var/www/app/Http/Controllers/RagController.php", "line": 38, "function": "answerQuestion", "class": "LLPhant\Query\SemanticSearch\QuestionAnswering", "type": "->" }, { "file": "/var/www/app/Http/Controllers/ChatBotController.php", "line": 12, "function": "initRag", "class": "App\Http\Controllers\RagController", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php", "line": 46, "function": "pergunta", "class": "App\Http\Controllers\ChatBotController", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Route.php", "line": 260, "function": "dispatch", "class": "Illuminate\Routing\ControllerDispatcher", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Route.php", "line": 206, "function": "runController", "class": "Illuminate\Routing\Route", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 808, "function": "run", "class": "Illuminate\Routing\Route", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 144, "function": "Illuminate\Routing\{closure}", "class": "Illuminate\Routing\Router", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php", "line": 50, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Routing\Middleware\SubstituteBindings", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php", "line": 88, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Foundation\Http\Middleware\VerifyCsrfToken", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php", "line": 49, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\View\Middleware\ShareErrorsFromSession", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php", "line": 121, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php", "line": 64, "function": "handleStatefulRequest", "class": "Illuminate\Session\Middleware\StartSession", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Session\Middleware\StartSession", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php", "line": 37, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Cookie\Middleware\AddQueuedCookiesToResponse", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php", "line": 75, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Cookie\Middleware\EncryptCookies", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 119, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 807, "function": "then", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 786, "function": "runRouteWithinStack", "class": "Illuminate\Routing\Router", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 750, "function": "runRoute", "class": "Illuminate\Routing\Router", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 739, "function": "dispatchToRoute", "class": "Illuminate\Routing\Router", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php", "line": 201, "function": "dispatch", "class": "Illuminate\Routing\Router", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 144, "function": "Illuminate\Foundation\Http\{closure}", "class": "Illuminate\Foundation\Http\Kernel", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php", "line": 21, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php", "line": 31, "function": "handle", "class": "Illuminate\Foundation\Http\Middleware\TransformsRequest", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Foundation\Http\Middleware\ConvertEmptyStringsToNull", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php", "line": 21, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php", "line": 51, "function": "handle", "class": "Illuminate\Foundation\Http\Middleware\TransformsRequest", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Foundation\Http\Middleware\TrimStrings", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Http/Middleware/ValidatePostSize.php", "line": 27, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Http\Middleware\ValidatePostSize", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php", "line": 110, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Foundation\Http\Middleware\PreventRequestsDuringMaintenance", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Http/Middleware/HandleCors.php", "line": 49, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Http\Middleware\HandleCors", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Http/Middleware/TrustProxies.php", "line": 57, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Http\Middleware\TrustProxies", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 119, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php", "line": 176, "function": "then", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php", "line": 145, "function": "sendRequestThroughRouter", "class": "Illuminate\Foundation\Http\Kernel", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Application.php", "line": 1188, "function": "handle", "class": "Illuminate\Foundation\Http\Kernel", "type": "->" }, { "file": "/var/www/public/index.php", "line": 17, "function": "handleRequest", "class": "Illuminate\Foundation\Application", "type": "->" } ] }

[f-lombardo]

Hi @Gudetomi , first of all you need to install Ollama locally https://ollama.com/ (you can also run Ollama via docker, but I think that it's easier and faster to run a local installation of it).

After installation, you can verify that Ollama is correctly installed by running:

ollama

You should see the various options of the command.

Then you need to download a model, for example:

ollama pull llama2

Once the model is downloaded, you can run it with the following command:

ollama run llama2

Then you can run your PHP application with LLPhant:

$config = new OllamaConfig();
$config->model = 'llama2';
$chat = new OllamaChat($config);

[Gudetomi]

Hi @f-lombardo , I ran Ollama via docker, trying to use llama3.1 but this error occurs.

[Gudetomi]

@f-lombardo Would there be any possibility of using GroqClud?

[f-lombardo]

@f-lombardo Would there be any possibility of using GroqClud?

From a technical point of view, I don't see problems. Try to create an issue for that, so we can verify how many people are interested in it and we can understand if anyone wants to create an implementation.

[f-lombardo]

Hi @f-lombardo , I ran Ollama via docker, trying to use llama3.1 but this error occurs.

You could try these steps:

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

docker exec -it ollama ollama run llama3.1

[f-lombardo]

Hi @Gudetomi ,

Depending on wheter you are running your PHP application and your Ollama server, there could be various settings for the url properties of the OllamaConfig:

Ollama	PHP	url
local	local	http://localhost:11434/api/
local	docker	http://host.docker.internal:11434/api/
docker	local	http://localhost:11434/api/
docker	docker	http://ollama:11434/api/

In the above schema I supposed that:

1. If Ollama runs in a docker container, it exposes the 11434 port

   docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

2. If both ollama and PHP are running inside docker containers, they are started with docker compose and the Ollama service is called ollama

[f-lombardo]

Depending on wheter you are running your PHP application and your Ollama server, there could be various settings for the url properties of the OllamaConfig:

Hi @Gudetomi , did this suggestion fix your problem?

[Gudetomi]

@f-lombardo I was limited in GPU memory to use Ollama, as it is for academic work I will use Mistral as it uses the model online. Thank you for your attention!