Does it support the Ollama service?

[Leroy-X]

Does this plugin support Ollama?

I attempted to modify the URL to http://localhost:11434/v1/chat/completions, but it failed to function.

What should I do? Thank you immensely.

[Krazal]

Hi!

Thank you for your question and your patience!

Unfortunately the plugin NppOpenAI doesn't support Ollama directly, but you can use a PHP proxy to handle NppOpenAI requests and Ollama responses. For example you can create an ollama_proxy.php:

<?php
header('Content-Type: application/json; charset=utf-8');

// Set up some variables
$ollama_url = 'http://localhost:11434/v1/chat/completions';
$postfields = [ # 'prompt' and (optional) 'system' will be added later
    'model'   => 'latest', # PLEASE SET UP MODEL HERE! (additional models: 'llama2', 'orca-mini:3b-q4_1', 'llama2:13b' etc.)
    'stream'  => false,
    'options' => [
        'temperature' => 0.7, # Default -- will be overwritten by NppOpenAI.ini
        'top_k '      => 40,  # Default value
        'top_p'       => 0.9, # Default -- will be overwritten by NppOpenAI.ini
        // For additional options see: https://github.com/ollama/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values
    ],
];

// Let's get started
try {

    // Check PHP Input
    $input = json_decode(file_get_contents('php://input'), true);
    if (!$input)
    {
        throw new \Exception("Non-JSON input received");
    }

    // Check system/prompt related elements
    $is_system_message = (@$input['messages'][0]['role'] == 'system');
    if (!isset($input['messages'][0]['content']))
    {
        throw new \Exception("No message received");
    }
    if ($is_system_message && !isset($input['messages'][1]['content']))
    {
        throw new \Exception("No message received, only instructions (system message)");
    }

    // Add system message
    if ($is_system_message)
    {
        $postfields['system'] = $input['messages'][0]['content'];
    }

    // Add prompt
    $postfields['prompt'] = !$is_system_message
        ? $input['messages'][0]['content']
        : $input['messages'][1]['content'];

    // Update some options, if possible
    // $postfields['model']               = $input['model']       ?? $postfields['model']; # Use the model above to support system messages
    $postfields['options']['temperature'] = $input['temperature'] ?? $postfields['options']['temperature'];
    $postfields['options']['top_p']       = $input['top_p']       ?? $postfields['options']['top_p'];

    // Call Ollama
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $ollama_url);
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0); # OK on localhost
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0); # OK on localhost
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); # It may come in handy
    curl_setopt($ch, CURLOPT_MAXREDIRS, 10);     # It may come in handy
    curl_setopt($ch, CURLOPT_TIMEOUT, 60);       # Increase if necessary
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); # Required for output
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($postfields));
    $curl_data   = curl_exec($ch);
    $curl_errno = curl_errno($ch);
    if ($curl_errno)
    {
        $curl_err = curl_error($ch) ?: $curl_errno;
        curl_close($ch);
        throw new \Exception("CURL error: {$curl_err}");
    }

    // Handle Ollama's response
    $response = json_decode($curl_data, true);
    if (!$response)
    {
        throw new \Exception("Non-JSON response received: {$curl_data}");
    }
    if (!isset($response['response']))
    {
        throw new \Exception("Missing response; Ollama's answer: " . print_r($response, true));
    }

    // Convert/Print output
    $output = [
        'usage'   => [
            'total_tokens' => (int)@$response['prompt_eval_count'] + (int)@$response['eval_count'], # Total token usage
        ],
        'choices' => [ [
            'message' => [
                'content'       => $response['response'],
                'finish_reason' => 'stop',
            ],
        ] ],
    ];
    echo json_encode($output);

// Handle errors
} catch (Exception $e) {
    echo json_encode([
        'error' => [
            'message' => $e->getMessage(),
        ],
    ]);
}

?>

You may install a WAMPServer and save the script above to C:\wamp64\www\ollama_proxy.php or similar directory.

Then you should update the API URL in NppOpenAI.ini (Plugins » NppOpenAI » Edit Config), like:

api_url=http://localhost/ollama_proxy.php?endpoint=
temperature=0.7
top_p=0.8

Please leave the model unchanged (e.g. model=gpt-4), and update this in the ollama_proxy.php file if necessary (look for the 'latest' keyword).

After saving the NppOpenAI.ini please don't forget to click Plugins » NppOpenAI » Load Config.

The script above should work, but it has been tested in simulated environment only, without Ollama. If you have any question, feel free to write a new Comment.

[Leroy-X]

Thank you very much for your reply!

[Zio3D]

Hi, I had problems with ollama and llama3.2 model. If anyone needs it, I made some changes to the "ollama_proxy.php" file and now it works.

ollama_proxy.php:

<?php
 header('Content-Type: application/json; charset=utf-8');

 if ($_SERVER['REQUEST_METHOD'] !== 'POST') {
  echo "{ \"Warning\": \"Missing 'POST' data.\" }";
  exit;
 }

 if ($_GET['endpoint'] != '/v1/chat/completions') {
  echo "\$_GET['endpoint'] != '/v1/chat/completions'";
  exit;
 }

 // Check PHP Input
 $input = json_decode(file_get_contents('php://input'), true);
 if (!$input)
 {
     throw new \Exception("Non-JSON input received");
 }

 // Set up some variables
 $ollama_url = 'http://localhost:11434/v1/chat/completions';
 $postfields = [
     'model'   => 'llama3.2',
     "messages" => [
         [
             "role" => "user",
             "content" => $input['messages']['0']['content']
         ]
     ],
     'stream'  => false,
     'options' => [
         'temperature' => 0.7, # Default -- will be overwritten by NppOpenAI.ini
         'top_k '      => 40,  # Default value
         'top_p'       => 0.9, # Default -- will be overwritten by NppOpenAI.ini
         // For additional options see: https://github.com/ollama/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values
     ],
 ];

 // Let's get started
 // Check system/prompt related elements
 $is_system_message = (@$input['messages'][0]['role'] == 'system');
 if (!isset($input['messages'][0]['content']))
 {
     throw new \Exception("No message received");
 }
 if ($is_system_message && !isset($input['messages'][1]['content']))
 {
     throw new \Exception("No message received, only instructions (system message)");
 }

 // Add system message
 if ($is_system_message)
 {
     $postfields['system'] = $input['messages'][0]['content'];
 }

 // Add prompt
 $postfields['prompt'] = !$is_system_message
     ? $input['messages'][0]['content']
     : $input['messages'][1]['content'];

 // Update some options, if possible
 // $postfields['model']               = $input['model']       ?? $postfields['model']; # Use the model above to support system messages
 $postfields['options']['temperature'] = $input['temperature'] ?? $postfields['options']['temperature'];
 $postfields['options']['top_p']       = $input['top_p']       ?? $postfields['options']['top_p'];

 // Call Ollama
 $ch = curl_init();
 curl_setopt($ch, CURLOPT_URL, $ollama_url);
 curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0); # OK on localhost
 curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0); # OK on localhost
 curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); # It may come in handy
 curl_setopt($ch, CURLOPT_MAXREDIRS, 10);     # It may come in handy
 curl_setopt($ch, CURLOPT_TIMEOUT, 60);       # Increase if necessary
 curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); # Required for output
 curl_setopt($ch, CURLOPT_POST, 1);
 curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($postfields));
 $curl_data   = curl_exec($ch);
 $curl_errno = curl_errno($ch);
 if ($curl_errno)
 {
     $curl_err = curl_error($ch) ?: $curl_errno;
     curl_close($ch);
     throw new \Exception("CURL error: {$curl_err}");
 }

 // Handle Ollama's response
 $response = json_decode($curl_data, true);

 if (!$response)
 {
     throw new \Exception("Non-JSON response received: {$curl_data}");
 }

 // Estrai i valori necessari
 $total_tokens = (int)$response['usage']['total_tokens'];
 $content = $response['choices'][0]['message']['content'];

 // Crea il nuovo JSON di risposta
 $new_response = [
     "usage" => [
         "total_tokens" => $total_tokens
     ],
     "choices" => [
         [
             "message" => [
                 "content" => $content,
                 "finish_reason" => "stop"
             ]
         ]
     ]
 ];
 echo json_encode($new_response); 
?>

The "NppOpenAI.ini" configuration file is set as:

[API]
secret_key=ollama-does-not-need-api-key
model=gpt-4o-mini
temperature=0.7
max_tokens=0
top_p=0.8
frequency_penalty=0
presence_penalty=0
#api_url=https://api.openai.com/
api_url=http://localhost/ollama_proxy.php?endpoint=
[PLUGIN]
keep_question=1
total_tokens_used=987
is_chat=0

In this configuration file it is written "model=gpt-4o-mini" but in reality the "llama3.2" model is used (it is set in the "ollama_proxy.php" file). In "NppOpenAI.ini", I only changed the model to "gpt-4o-mini" (no api key needed).