Llama2 Prompt Engineering

[sdg4168]

[ Hi I'm studying about llama2. I'm trying to create a chat bot using llama open source, and My goal is to receive accurate answers when asked about embedded data. A query engine is built by embedding external data in the RAG system created through LlamaIndex. I'm also attempting prompt engineering using Few-shot Prompting, CoT (Chain of Thought), and Self-Consistentcy. ]

[ Here's the problem. If I insert an example of a date in the prompt and ask a question about a single date (ex: 2023-10-19 14:48:29), the model will answer well. However, if I insert several examples in the prompt and ask about the date range (ex: 2023-10-19 14:40:00 ~ 2023-10-19 15:00:00), llama will get an incorrect answer. ]

---

When what llama provides is accurate information, the prompt written is as follows: ↓↓↓↓

      Your goal is to provide answers relating to the system logs.
      You are a technical supporter who is helpful to the company.
      All you have to do is find what I'm asking for in the csv file embedded in you, read each item, and answer.
      My csv file is a log file consisting of Type, Occurred Time, IP Address, and Log Details.
      If you don't know the answer to a question, don't share false information.

      For example:
      user : Please tell me the log with occured time '2023-10-17 16:45:33'
      response : The log that occurred on 2023-10-17 16:45:33 is as follows:
      Type : Information
      Occurred Time : 2023-10-17 13:24:09
      IP Address : 192.~~
      Log Details : Unable to establish a connection to the database.

This is the answer given by llama2 : ↓↓↓↓

    Of course! I'm happy to help you with that. Based on the information provided in the CSV ile. the log with ...
    Type : Error
    Occured time : 2023-10-17 16:45:33
    IP Address : 192.~
    Log Details : SQL syntax error in query.
    I hope that helps! Let me know if you have any other questions.

It's a good example. That is actually correct information about embedded data.

---

When what llama provides is incorrect information, the prompt written is as follows: ↓↓↓↓

      Your goal is to provide answers relating to the system logs.

      You are a technical supporter who is helpful to the company.
      All you have to do is find what I'm asking for in the csv file embedded in you, read each item, and answer.
      My csv file is a log file consisting of Type, Occurred Time, IP Address, and Log Details.
      If you don't know the answer to a question, don't share false information.

      For example 1 :
      user : Please tell me the log with occured time '2023-10-17 16:45:33'
      response : The log that occurred on 2023-10-17 16:45:33 is as follows:
      Type : Error
      Occurred Time : 2023-10-17 16:45:33
      IP Address : 192.~~
      Log Details : Unable to establish a connection to the database.

      For example 2 :
      user :Please tell me all the logs whose type is warning and the occurrence time is October 19, 2023.
      reponse : There are 3 in total. Here are the details of the warning log that occurred on October 19th.
      Type : Warning
      Occured Time : 2023-10-19 14:12:23
      P Address : 192.~~
      Log Details : Low disk space detected on drive C:\.
      Type : Warning
      Occured Time : 2023-10-19 15:47:51
      IP Address : 192.~~
      Log Details : Low disk space detected on drive C:\.
      Type : Warning
      Occured Time : 2023-10-19 14:48:38
      IP Address : 192.~~
      Log Details : Low disk space detected on drive C:\.

This is the answer given by llama2 : ↓↓↓↓

    Based on the information provided in the CSV file and the context information, there are 8 logs with a type of Warning and an occurrence time of October 18, 2023. Here are the details of those logs:
    1. Occurred Time: 2023-10-18 10:32:43
    IP Address: 10.3.1.136
    Log Details: Multiple failed login attempts
    2. Occurred Time: 2023-10-18 11:23:51
    IP Address: 10.3.1.136
    Log Details: Multiple failed login attempts

It's a wrong example. That is actually incorrect information about embedded data.

---

[ To conclude, I wonder if llama2 is not good at recognizing dates. It embedded data containing dates and showed examples in the prompt, but it doesn't seem to understand date ranges other than single dates. For example, When I asked to know all the logs that occurred on October 19th, if there were actually 8 logs that occurred, I would like them to answer all 8. ]

What do I need??

I use that model : meta-llama/Llama-2-7b-chat-hf.

[Vatsal1106Virani]

I have same issue I have created one system prompt and give to model then model not give proper answer of that. My system prompt is about to generate color palettes for poster making particular for independence day of India and palette contains background, heading 1 and heading 2 color as per contrast.

[Jaswir]

Hi I am new to this opensource project. How does contributing here work? Are there people from Meta/Members that help with questions when people try to solve something and get blocked?

So far I haven't seen members in the open and closed issues I browsed. Also what are the expectations? What is like protocol in this Open Source project in terms of help you can expect from Members? Is it more like contributors are expected to do most themselves and I can expect a small hint here and there? Is it like lots of contributors working together and help each other out? No members involved?

[Jaswir]

@sdg4168 @Vatsal1106Virani Sorry if this stupid question I am a beginner Gen AI Enthusiast. How are you using this model? What are you using to use this? Google Collab?

[Vatsal1106Virani]

@sdg4168 @Vatsal1106Virani Sorry if this stupid question I am a beginner Gen AI Enthusiast. How are you using this model? What are you using to use this? Google Collab?

Follow The Instruction From README.md file from Below link. https://github.com/facebookresearch/llama

[Jaswir]

I have same issue I have created one system prompt and give to model then model not give proper answer of that. My system prompt is about to generate color palettes for poster making particular for independence day of India and palette contains background, heading 1 and heading 2 color as per contrast.

Hi Vatsal, What is the prompt ? What is the answer you get back?

This information will help solve the problem, thanks.

[sdg4168]

@sdg4168 @Vatsal1106Virani Sorry if this stupid question I am a beginner Gen AI Enthusiast. How are you using this model? What are you using to use this? Google Collab?

@Jaswir Hi Jaswir, I am using t4 gpu in google colab. This is a free version, and you can expect better performance if you use the paid version.

[Vatsal1106Virani]

I have same issue I have created one system prompt and give to model then model not give proper answer of that. My system prompt is about to generate color palettes for poster making particular for independence day of India and palette contains background, heading 1 and heading 2 color as per contrast.

Hi Vatsal, What is the prompt ? What is the answer you get back?

This information will help solve the problem, thanks.

You can Learn though Below Course.

https://courses.nvidia.com/courses/course-v1:DLI+S-FX-12+V1/

[Jaswir]

I have same issue I have created one system prompt and give to model then model not give proper answer of that. My system prompt is about to generate color palettes for poster making particular for independence day of India and palette contains background, heading 1 and heading 2 color as per contrast.

Hi Vatsal, What is the prompt ? What is the answer you get back? This information will help solve the problem, thanks.

You can Learn though Below Course.

https://courses.nvidia.com/courses/course-v1:DLI+S-FX-12+V1/

Wow thanks!

[Jaswir]

@Vatsal1106Virani @sdg4168

Maybe you should ask your question here: https://github.com/run-llama/llama_index It seems to be specifically focussed on llama index

[Vatsal1106Virani]

Hi @Jaswir ,

You are Participate in 48 hour Custom GPTs Hackathon. How is over all experience?

[Jaswir]

@Vatsal1106Virani

Very good. I met someone that I had a great team up with and learned a lot about Chat GPT plugins and it helped me get Chat GPT 4 subscription and bypass waiting list. I feel my AI knowledge increasing from each participation. I got a better understanding of finetuning gpt's from it. Also it's an accelerator program for startups, if you're interested in setting up a winning startup I think it's valuable also. I learned much about what to think about when creating a startup. Like Tam, Sam, Competitor Analysis.

I'll be joining this one too: https://lablab.ai/event/yi-llms-24-hours-hackathon?enroll=true Invite you to join too. But this one even better if you can make it: https://lablab.ai/event/truera-challenge-build-llm-applications You'll get free google cloud credits and stuff!

You followed that course? https://courses.nvidia.com/courses/course-v1:DLI+S-FX-12+V1/ Is it worth it?

[Vatsal1106Virani]

@Vatsal1106Virani

Very good. I met someone that I had a great team up with and learned a lot about Chat GPT plugins and it helped me get Chat GPT 4 subscription and bypass waiting list. I feel my AI knowledge increasing from each participation. I got a better understanding of finetuning gpt's from it. Also it's an accelerator program for startups, if you're interested in setting up a winning startup I think it's valuable also. I learned much about what to think about when creating a startup. Like Tam, Sam, Competitor Analysis.

I'll be joining this one too: https://lablab.ai/event/yi-llms-24-hours-hackathon?enroll=true Invite you to join too. But this one even better if you can make it: https://lablab.ai/event/truera-challenge-build-llm-applications You'll get free google cloud credits and stuff!

You followed that course? https://courses.nvidia.com/courses/course-v1:DLI+S-FX-12+V1/ Is it worth it?

Yes, It's very helpful.

[WuhanMonkey]

@sdg4168 We have some examples for LlamaIndex and RAG in Llama recipes repo here, can you take a look and let us know there?

[Jaswir]

@Vatsal1106Virani @WuhanMonkey

I am confused about the Bloke Models. What's the difference between: https://huggingface.co/TheBloke/LLaMa-7B-GGML

and https://huggingface.co/meta-llama/Llama-2-7b-chat-hf I am exploring Llama open source project to contribute, I wonder if someone has an issue about LLama-2-7b-chat-hf on the open source project and I use the bloke's fine tuned version will it provide the same errors as the original models? Can I use it to reproduce errors and such? And to bring solutions. Like if solution works for TheBloke model will it also work for meta-llama's original model?

Let me know the proper place to ask this if not here.

[WuhanMonkey]

@Vatsal1106Virani @WuhanMonkey

I am confused about the Bloke Models. What's the difference between: https://huggingface.co/TheBloke/LLaMa-7B-GGML

and https://huggingface.co/meta-llama/Llama-2-7b-chat-hf I am exploring Llama open source project to contribute, I wonder if someone has an issue about LLama-2-7b-chat-hf on the open source project and I use the bloke's fine tuned version will it provide the same errors as the original models? Can I use it to reproduce errors and such? And to bring solutions. Like if solution works for TheBloke model will it also work for meta-llama's original model?

Let me know the proper place to ask this if not here.

If your question is not related to this issue, then it is not the proper place to ask.

We also want to have this repo focus on issues with original Llama 2 models. For your type of question, it is better been asked in HF forum.