MaartenGr / BERTopic

Leveraging BERT and c-TF-IDF to create easily interpretable topics.

https://maartengr.github.io/BERTopic/

MIT License

6.14k stars 765 forks source link

Add system prompt for OpenAI representation model #2146

Open Leo-LiHao opened 1 month ago

Leo-LiHao commented 1 month ago

Feature request

Currently, the system prompt of the OpenAI representation model is fixed. This feature allows developers to customize the system prompt.

Motivation

Provide flexibility to developers to set their own system prompt for OpenAI models

Your contribution

2145 Added system prompt for OpenAI models

MaartenGr commented 1 month ago

Thank you for this feature request. If we consider adding system prompts as a parameter for these models, shouldn't we then also consider adding the same for all other LLMs? From a user experience perspective, if one model has that parameter, I would expect all others to also have that same functionality.

Leo-LiHao commented 1 month ago

Hi @MaartenGr, thank you for the feedback! Yes, it would be better to add the same for all other LLMs that support system prompts. If you'd like, I can add them.

MaartenGr commented 1 month ago

@Leo-LiHao If it doesn't take too much time, that would be great! I think it would be a nice user experience to have it available for all LLM-based representation models.

Leo-LiHao commented 1 month ago

@MaartenGr I checked the implemented LLM-based representation models:

Cohere: We're using the legacy generate API, which doesn't support system prompts. The newer chat API does support them, so we could consider migrating if needed.
LangChain: The system prompt is set when creating the chain, so the feature is already supported implicitly. We could update the documentation to include an example that uses a system prompt when configuring the chain, which could clarify this for users.
LlamaCPP: From what I understand, we used this library for text generation rather than chat, so system prompts aren’t supported.

Let me know if I missed anything or if you'd like to discuss further steps.

MaartenGr commented 1 month ago

Cohere: We're using the legacy generate API, which doesn't support system prompts. The newer chat API does support them, so we could consider migrating if needed.

Sounds good!

LangChain: The system prompt is set when creating the chain, so the feature is already supported implicitly. We could update the documentation to include an example that uses a system prompt when configuring the chain, which could clarify this for users.

Agreed, it should be clear to the users why there isn't a system prompt available whereas it is being used in other LLMs.

LlamaCPP: From what I understand, we used this library for text generation rather than chat, so system prompts aren’t supported.

Then it might be worthwhile to switch over to chat ompletion when we have a system prompt or use chat completion by default What do you think?

Leo-LiHao commented 1 month ago

@MaartenGr I migrated Cohere's generate to chat API (#2145).

However, I'm not familiar with LangChain and LlamaCPP. I'd appreciate it if someone can lend a hand :)