could lm studio and its implementation of open source locally run models work in Jarvis

[bongiben]

https://lmstudio.ai

it can provide an open api style

I am no expert just found both your project and this tonight

here's code they suggest for python

# Example: reuse your existing OpenAI setup
import os
import openai

openai.api_base = "http://localhost:1234/v1" # point to the local server
openai.api_key = "" # no need for an API key

completion = openai.ChatCompletion.create(
  model="local-model", # this field is currently unused
  messages=[
    {"role": "system", "content": "Always answer in rhymes."},
    {"role": "user", "content": "Introduce yourself."}
  ]
)

print(completion.choices[0].message)

[alondmnt]

yes, this (theoretically) should work already in the current version of Jarvis. using the example that you provided, the corresponding settings in Jarvis will be:

set Chat: Model to: (online) OpenAI or compatible: custom model

then, in the advanced settings section of Jarvis:

set Chat: OpenAI (or compatible) custom model ID to: local-model

check this option: Chat: Custom model is a conversation model

and finally, set Chat: Custom model API endpoint to: http://localhost:1234/v1/chat/completions

[bongiben]

I was able to enter those parameters but it is coming back with an error

(In plugin: Jarvis)

Please specify a valid OpenAI API key in the settings

[alondmnt]

could you try entering a random string in the setting Model: OpenAI API Key?

[bongiben]

works fine, didn't didn't need a token set I didnt follow your code exactly e.g. #and finally, set Chat: Custom model API endpoint to: http://localhost:1234/v1/chat/completions reentered the information and it works a treat thanks so much, you should get a video out about this on YouTube, would be very useful for people!