MaartenGr / KeyBERT

Minimal keyword extraction with BERT
https://MaartenGr.github.io/KeyBERT/
MIT License
3.46k stars 344 forks source link

Fail to parse OpenAI api response #212

Open lucafirefox opened 6 months ago

lucafirefox commented 6 months ago

I'm using keybert to run some topic modeling tasks and it's very powerful. However, I did have to make some changes to the source code to make the code working flawlessly. Here is my code:

import openai
from keybert.llm import OpenAI
from keybert import KeyLLM, KeyBERT

# Create your LLM 
client = openai.OpenAI(api_key="sk-XXXXXXXXX")
llm = OpenAI(client, prompt=CUSTOM_PROMPT)

# Load it in KeyLLM
kw_model = KeyBERT(llm=llm)

# Extract keywords
keywords = kw_model.extract_keywords(documents)

which throws the following error:

AttributeError: 'CompletionChoice' object has no attribute 'message'

Here below a snippet of the traceback:

Screenshot 2024-03-11 at 16 19 43

I believe OpenAI response changed over time and my working work-around is to replace:

keywords = response.choices[0].message.content.strip()

with

keywords = response.choices[0].text.strip()

in file keybert/llm/_openai.py at lines 181 and 189

MaartenGr commented 6 months ago

Thanks for sharing this. OpenAI migrated to a new major release a while ago, so it seems it still needs to be updated here. If you want, a PR would be highly appreciated.

lucafirefox commented 6 months ago

PR done, feel free to close the issue!