imoneoi / openchat

OpenChat: Advancing Open-source Language Models with Imperfect Data
https://openchat.team
Apache License 2.0
5.26k stars 398 forks source link

Not handling the decimal point values correctly #89

Open pradeepdev-1995 opened 1 year ago

pradeepdev-1995 commented 1 year ago

I have an input ex: sales=1893.34, also I am giving prompts to perform certain tasks but the openchat returns such as sales=189.334 or sales=18933.4 in the response. The decimal point is moving left and right. so it deviates from the actual value. How to solve this problem via prompting?

imoneoi commented 1 year ago

Can you try a lower temperature (like 0.0 and 0.5) first?

pradeepdev-1995 commented 1 year ago

Tried this. But still getting the issue

imoneoi commented 1 year ago

Can you post the prompt here?

pradeepdev-1995 commented 1 year ago

Here is the sample data:(question,keys and values are not real)

system_msg = """As an AI programmed to generate business report from JSON input data,
input: {'abc: 12.05, 'def': 96181.4593197, 'ghi': True}
question:  sample question
report:"""
imoneoi commented 1 year ago

The system message may not work properly. Try putting it to the first user turn? I've tested it and seems working fine (temp = 0.5)

Generate annual business report and analysis from the JSON data in Markdown. use tables when appropriate

data: {
  "report": {
    "january": {
      "revenue": 34567.89,
      "expenses": 12345.67,
      "profit": 22222.22,
      "customers": 150,
      "orders": 500,
      "average_order_value": 69.87
    },
    "february": {
      "revenue": 45678.90,
      "expenses": 13456.78,
      "profit": 32222.12,
      "customers": 175,
      "orders": 550,
      "average_order_value": 81.02
    },
    "march": {
      "revenue": 56789.01,
      "expenses": 14567.89,
      "profit": 42222.12,
      "customers": 190,
      "orders": 600,
      "average_order_value": 92.13
    },
    "april": {
      "revenue": 67890.12,
      "expenses": 15678.90,
      "profit": 52222.22,
      "customers": 205,
      "orders": 650,
      "average_order_value": 101.32
    },
    "may": {
      "revenue": 78901.23,
      "expenses": 16578.91,
      "profit": 62222.32,
      "customers": 215,
      "orders": 700,
      "average_order_value": 112.04
    },
    "june": {
      "revenue": 89012.34,
      "expenses": 17578.92,
      "profit": 71222.42,
      "customers": 225,
      "orders": 750,
      "average_order_value": 121.16
    },
    "july": {
      "revenue": 99013.45,
      "expenses": 18578.93,
      "profit": 80222.52,
      "customers": 235,
      "orders": 800,
      "average_order_value": 130.27
    },
    "august": {
      "revenue": 109014.56,
      "expenses": 19578.94,
      "profit": 89222.62,
      "customers": 245,
      "orders": 850,
      "average_order_value": 132.49
    },
    "september": {
      "revenue": 119015.67,
      "expenses": 20578.95,
      "profit": 98222.72,
      "customers": 255,
      "orders": 900,
      "average_order_value": 135.12
    },
    "october": {
      "revenue": 129016.78,
      "expenses": 21578.96,
      "profit": 107222.82,
      "customers": 265,
      "orders": 950,
      "average_order_value": 138.95
    },
    "november": {
      "revenue": 139017.89,
      "expenses": 22578.97,
      "profit": 106222.92,
      "customers": 275,
      "orders": 1000,
      "average_order_value": 144.03
    },
    "december": {
      "revenue": 149019.01,
      "expenses": 23578.98,
      "profit": 125222.03,
      "customers": 285,
      "orders": 1050,
      "average_order_value": 147.62
    }
  }
}

report:

Annual Business Report and Analysis

Table of Contents

  1. Revenue
  2. Expenses
  3. Profit
  4. Customers
  5. Orders
  6. Average Order Value

Revenue

Month Revenue
January $34,567.89
February $45,678.90
March $56,789.01
April $67,890.12
May $78,901.23
June $89,012.34
July $99,013.45
August $109,014.56
September $119,015.67
October $129,016.78
November $139,017.89
December $149,019.01

Expenses

Month Expenses
January $12,345.67
February $13,456.78
March $14,567.89
April $15,678.90
May $16,578.91
June $17,578.92
July $18,578.93
August $19,578.94
September $20,578.95
October $21,578.96
November $22,578.97
December $23,578.98

Profit

Month Profit
January $22,222.22
February $32,222.12
March $42,222.12
April $52,222.22
May $62,222.32
June $71,222.42
July $80,222.52
August $89,222.62
September $98,222.72
October $107,222.82
November $106,222.92
December $125,222.03

Customers

Month Customers
January 150
February 175
March 190
April 205
May 215
June 225
July 235
August 245
September 255
October 265
November 275
December 285

Orders

Month Orders
January 500
February 550
March 600
April 650
May 700
June 750
July 800
August 850
September 900
October 950
November 1000
December 1050
pradeepdev-1995 commented 1 year ago

@imoneoi I am formating the prompt below

<s>[IDENTITY]
As an AI programmed to generate business reports from JSON input data,
[/IDENTITY]</s>
[INST]
Now Based on the following input: {'abc': 12.05, 'def': 96181.4593197, 'ghi': True}
Generate the report for this Question: sample question
Report:
 [/INST]

Is there any worng in this method of prompt structure?Also if i should pass any few shot examples,where can I pass those?

imoneoi commented 1 year ago

OpenChat is using a special prompt format, which you can see it here. You can pass few-shot examples in the first user message as well.

pradeepdev-1995 commented 1 year ago

i could see

GPT4 Correct User: Hello<|end_of_turn|>GPT4 Correct Assistant: Hi<|end_of_turn|>GPT4 Correct User: How are you today?<|end_of_turn|>GPT4 Correct Assistant:

In the mentioned page.

How does this fit in my context? @imoneoi Please share the structure based my current prompt - -<s>[IDENTITY] As an AI programmed to generate business reports from JSON input data, [/IDENTITY]</s> [INST] Now Based on the following input: {'abc': 12.05, 'def': 96181.4593197, 'ghi': True} Generate the report for this Question: sample question Report: [/INST]

imoneoi commented 1 year ago

Try putting the question to the first user turn

GPT4 Correct User: Question<|end_of_turn|>GPT4 Correct Assistant:
pradeepdev-1995 commented 1 year ago

@imoneoi I put in the way that you mentiond. still i am facing the decimal point movement issue in the response. What might be the issue? is openchat is not goot at dealing with mathmatics and decimal point values?