Migrate to gpt-3.5-turbo

[Olen]

I tried to change the model from the default text-davinci-003 to gpt-3.5-turbo But it seems like that model is not usable for the integration:

'This is a chat model and not supported in the v1/completions endpoint. Did you mean to use v1/chat/completions?' error_param=model error_type=invalid_request_error message='OpenAI API error received' 

...

openai.error.InvalidRequestError: This is a chat model and not supported in the v1/completions endpoint. Did you mean to use v1/chat/completions?

Now, I don't know enough about the different models and endpoints, so it might be that the "chat" endpoint can't be used, but it looks from the documentation as if the chat endpoint and gpt-3.5-turbo is the best option:

Chat vs Completions Because gpt-3.5-turbo performs at a similar capability to text-davinci-003 but at 10% the price per token, we recommend gpt-3.5-turbo for most use cases.

[drogfild]

Hi @Olen and all others!

I was able to mock this to work with gpt-3.5-turbo.

Modify sensor.py, replace generate_openai_response_sync with following:

def generate_openai_response_sync(model, prompt, temperature, max_tokens, top_p, frequency_penalty, presence_penalty):
    return openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "You are a helpful assistant with ironical responses."},
            {"role": "user", "content": prompt}
    ]
)

And much further down there's line for self._response_text. Replace with: self._response_text = response['choices'][0]['message']['content']

I'll try to make PR also if I have time for that.

[CajuCLC]

Hi @Olen and all others!

I was able to mock this to work with gpt-3.5-turbo.

Modify sensor.py, replace generate_openai_response_sync with following:

def generate_openai_response_sync(model, prompt, temperature, max_tokens, top_p, frequency_penalty, presence_penalty):
    return openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "You are a helpful assistant with ironical responses."},
            {"role": "user", "content": prompt}
    ]
)

And much further down there's line for self._response_text. Replace with: self._response_text = response['choices'][0]['message']['content']

I'll try to make PR also if I have time for that.

Thanks for this. I changed on my haas, but get this error:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 304, in _async_setup_platform
    await asyncio.shield(task)
  File "/config/custom_components/openai_response/sensor.py", line 28, in async_setup_platform
    async_add_entities([OpenAIResponseSensor(hass, name, model)], True)
  File "/config/custom_components/openai_response/sensor.py", line 47, in __init__
    self._response_text = response['choices'][0]['message']['content']
NameError: name 'response' is not defined

[drogfild]

Hi @CajuCLC,

Maybe there's just some typo in the code. Here's the whole async_generate_openai_response function from my test code. Hope this helps. Remember to reboot whole HA after modifying. Not totally sure if needed, but worth to try.

    async def async_generate_openai_response(self, entity_id, old_state, new_state):
        new_text = new_state.state
        if new_text:
            response = await self._hass.async_add_executor_job(
                generate_openai_response_sync,
                self._model,
                new_text,
                0.9,
                964,
                1,
                0,
                0
            )
#            self._response_text = response["choices"][0]["text"]
            self._response_text = response['choices'][0]['message']['content']
            self._state = "response_received"
            self.async_write_ha_state()

[CajuCLC]

Nice. That worked. I am also using gpt-4-32k model. :)

[Olen]

I just started looking at this and thought about creating a PR, but I really think the system content should be customizable as well.

            {"role": "system", "content": "You are a helpful assistant with ironical responses."},

Not everyone will want a "helpful assistant with ironical responses". But that requires a way to choose the "tone" of the reponse as well as the actual request.

• To simplify that, I think I'll add the service call I (#3) in the same PR.

[drogfild]

I totally agree. That was only my example. System content makes a great difference how AI generates it's answers. In fact I already changed mine to "You are an announcement system on a wall. You are located in a family house in the kitchen. You make delightfull messages mostly in finnish.".

Additionally in my code the model is fixed in the code. That also needs to be dynamic. And proper code would need to check model user has selected to handle different completion (completions vs chatcompletions) based on that. Or alternatively only support newer models..

Service call would be a proper way to implement this type of call. Great to see this progresses :)

[Olen]

I have a first PoC here: https://github.com/Olen/openai_response/tree/gpt-3.5-turbo

It works, but I do get some weird errors, though they could be because I track the "dev" branch of HA in my development environment.

It adds a service-call:

service: openai.openai_input
data:
  prompt: Tell John that he has to get ready for school
  mood: You are a personal assistant with lots of humor

[Olen]

Please have a look at #6 It works fine here, and resolves this issue.

[drogfild]

Good work @Olen. Service call works perfectly! I would just add GPT-4 also for the options. Is it enough to add it to services.yaml selector like this? Cannot test it myself as I don't yet have that beta...

    model:
      name: Model
      advanced: true
      description: The model to use
      required: false
      example: "gpt-3.5-turbo"
      default: "gpt-3.5-turbo"
      selector:
        select:
          options:
            - "gpt-3.5-turbo"
            - "gpt-4"

[CajuCLC]

I have the beta if you guys want me to test it. I am actually running with gpt-4 right now.

sensor:
  - platform: openai_response
    api_key: XXX
    model: "gpt-4-32k" # Optional, defaults to "text-davinci-003"
    name: "hassio_openai_response" # Optional, defaults to "hassio_openai_response"

[Olen]

Yes. You can just add it to the yaml-file. The only issue with that is that gpt-4 is not publicly available (yet), and it might be confusing if people believe they can use it without signing up for it first.

The yaml-file only influence the GUI section of the service call. You can use whatever mode you want in manual trigger of the service call. So you can still use gpt-4-32k with #6 - it just can't be selected in the GUI.