search

alkem-io / server

Core server in the Alkemio platform, offering a GraphQL api for interacting with the logical domain model.
http://alkem.io
European Union Public License 1.2
24 stars 4 forks source link

VC gives OpenAI answers #4110

Closed larssondenise closed 1 month ago

larssondenise commented 1 month ago

Description

As a user, I want to ask my VC questions based on my content. However, it now gives me answers based on OpenAI but says that the source is the Subspace. Even with my empty subspace, the VC gives me answers. This scares me as a user and doesn't allow me to gain the right answers.

I want my VC to give me answers based on its data, or explicitly tell me that it is based on external sources.

Acceptance criteria

Additional Context

This VC is based on one document Marketing Assignment informatie document. However when I ask to give me a summary of the information in that document. This is the answer.

Hmm, I am not sure what specific information you are referring to in the 'Marketing Assignment informatie document'. However, as a Virtual Contributor, I can tell you that it is possible to use information from external documents to inform your marketing strategy. It is important to ensure that the information is relevant, accurate, and up-to-date. When using external information, it is also important to properly cite your sources and avoid plagiarism.

Another example: VC who is trained on the data by mazzucato

image.png

a vc trained on totally different data

msedge_Egswe8Ub6m.png

an empty vc trained on zero data

msedge_4MrU7lXdhf.png

Areas that will be affected

To be added during the refinement

larssondenise commented 1 month ago

@techsmyth

valentinyanakiev commented 1 month ago

@larssondenise I believe we need some educational materials about what is really expected from GenAI solution such as OpenAI. It has vast array of knowledge and I would not be surprised if book A, B and C are already ingested there. It has to be really specific, unique knowledge to make the answer reasonably different. I agree that the references could be way better - we show the most relevant sources, but most relevant does not really equate to relevant

larssondenise commented 1 month ago

@valentinyanakiev not quite sure what you mean... let's have a chat when you're back!