search

CodingTrain / Bizarro-Devin

12 stars 4 forks source link

Reworking the prompt #93

Closed shiffman closed 3 months ago

shiffman commented 3 months ago

Adding biographical information and trying to provide better context for the context?

kfahn22 commented 3 months ago

I don't think the biographical info is necessary. There is a Wikipedia page with most of this info, so the model will most likely already know it. Also, I would tell the model not to use the quotes verbatim. (I created a bot on HuggingFace and added quotes as context and this was a issue.)

supercrafter100 commented 3 months ago

OpenAI suggests using the quotes in their documentation for providing examples. During previous tests using them it didn't seem to even include them in the response so I'm guessing the model (gpt-4, llama) knows not to include them? I guess it can't hurt having it in the system prompt though.

shiffman commented 3 months ago

I don't think the biographical info is necessary. There is a Wikipedia page with most of this info, so the model will most likely already know it. Also, I would tell the model not to use the quotes verbatim. (I created a bot on HuggingFace and added quotes as context and this was a issue.)

It's not necessary but it's helpful and useful in my experience! I've run simple tests asking information about things like my bio or Nature of Code with and without additional context and the responses are more reliable and focus on the proper references and info with the additional information and context!

kfahn22 commented 3 months ago

I think you are right that adding background info can help, especially when it is likely to hallucinate. Are you planning on asking about his background?

On Sat, Apr 13, 2024 at 9:19 AM Daniel Shiffman @.***> wrote:

I don't think the biographical info is necessary. There is a Wikipedia page with most of this info, so the model will most likely already know it. Also, I would tell the model not to use the quotes verbatim. (I created a bot on HuggingFace and added quotes as context and this was a issue.)

It's not necessary but it's helpful and useful in my experience! I've run simple tests asking information about things like my bio or Nature of Code with and without additional context and the responses are more reliable and focus on the proper references and info with the additional information and context!

— Reply to this email directly, view it on GitHub https://github.com/CodingTrain/Bizarro-Devin/pull/93#issuecomment-2053644553, or unsubscribe https://github.com/notifications/unsubscribe-auth/APQ2Y4WFSWJ7SPWWJ77WSCLY5EWE5AVCNFSM6AAAAABGFBP2WWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJTGY2DINJVGM . You are receiving this because you commented.Message ID: @.***>