nhaouari / obsidian-textgenerator-plugin

Text Generator is a versatile plugin for Obsidian that allows you to generate text content using various AI providers, including OpenAI, Anthropic, Google and local models.
https://text-gen.com/
MIT License
1.47k stars 130 forks source link

Not providing the same answer, as compared to GPT3.5 or GPT4 #163

Closed tombrau closed 10 months ago

tombrau commented 1 year ago

I've been experimenting with some prompts for myself, I'm using the CTRL - J, by highlighting and pressing ctrl-j. Here are the experiments. As you can see GPT35 and GPT4 are in close agreement, however the plugin is out by a bit. Maybe I'm doing something wrong. I'm not to fussed by the difference in numbers, just by the difference in response, where some things are not reported and some things are wrong. Model is set for gpt4, tokens: 2048, temp: 0

Test prompt: You will act as my carb calculator. You will help me track the carbs in the foods I'm eating. I will provide the food items I have consumed along with the quantity and amount in grams or milliliters using the following format: "Food, quantity, amount (g or ml)" For example: "Chicken breast, 1, 100g: Bread,2,50g: End" After that you will compile the information and give me a md table with columns for food, quantity, amount, total, and carbs. The "Total" column will be the total weight of each food item in grams or milliliters, and the "carbs" column will show the number of carbohydrates I have consumed. Once the table is complete, You will provide me with the grand total of carbs for all the food items I have consumed, in the format of "Total Carbs Consumed: x amount" and nothing else. here is what i have consumed, " white bread, 4, 43 g:
Tomatoe,1,97 g: Cheese, 2, 30 g: Bell Peppers with cream cheess,1,100 g: dougnuts,1,70 g: milk,1,250 ml: End"

GPT3.5 Response

Food Quantity Amount (g/ml) Total (g/ml) Carbs (g)
White bread 4 43g 172g 68.24g
Tomato 1 97g 97g 4.53g
Cheese 2 30g 60g 1.2g
Bell peppers with cream cheese 1 100g 100g 6.0g
Doughnut 1 70g 70g 25.9g
Milk 1 250ml 250ml 12.5g

Total Carbs Consumed: 118.37g

GPT4 Response

Food Quantity Amount (g or ml) Total Carbs (g)
White bread 4 43 g 172 g 48.8
Tomato 1 97 g 97 g 4.2
Cheese 2 30 g 60 g 1.8
Bell Peppers with cream cheese 1 100 g 100 g 8.0
Doughnuts 1 70 g 70 g 31.5
Milk 1 250 ml 250 ml 12.0
Total Carbs Consumed 106.3

Total Carbs Consumed: 106.3 g

Obsidian Response

You will act as my carb calculator. You will help me track the carbs in the foods I'm eating. I will provide the food items I have consumed along with the quantity and amount in grams or milliliters using the following format: "Food, quantity, amount (g or ml)" For example: "Chicken breast, 1, 100g: Bread,2,50g: End" After that you will compile the information and give me a md table with columns for food, quantity, amount, total, and carbs. The "Total" column will be the total weight of each food item in grams or milliliters, and the "carbs" column will show the number of carbohydrates I have consumed. Once the table is complete, You will provide me with the grand total of carbs for all the food items I have consumed, in the format of "Total Carbs Consumed: x amount" and nothing else. here is what i have consumed, " white bread, 4, 43 g:
Tomatoe,1,97 g: Cheese, 2, 30 g: Bell Peppers with cream cheess,1,100 g: dougnuts,1,70 g: milk,1,250 ml: End"

[!ai]+ AI

Food Quantity Amount (g or ml) Total Carbs (g)
White bread 4 43 g 172 g 52.8
Tomato 1 97 g 97 g 4.2
Cheese 2 30 g 60 g 0.6
Bell Peppers with cream cheese 1 100 g 100 g 7
Doughnuts 1 70 g - -
Milk - - - -

Total Carbs Consumed: x amount Note: The carb values for doughnuts and milk are missing, so the total carbs consumed cannot be calculated accurately. Please provide the carb values for these items to get an accurate total.

Plexipop commented 1 year ago

Check the Default model parameters and align with those used by OpenAI. The latter do not seem to be totally clear though. This issues it broadly discussed also elsewhere.