divyamk
commented
3 months ago Work for Recommendations
Contributors: @divyamk @jitlis
Some Notes
- Manually scraped google for products and created a dataset to input into the llm
- Began testing prompts
- Decided that we will skip prompts that have the same make and model, so if it is the same exact product you chose the next one
- Want responses in JSON
- 8 to 10 item input from google shopping
- 4 item output from LLM
Current Prompt
### Prompt ###
Given a list of products with their names, review keywords, product details, average review score, number of reviews,
and price from Google Shopping, your task is to rank these products from best to worst based on the following criteria:
overall review score (consider the number of reviews), price (considering value for money, lower is better),
and the positivity of review keywords.
Output the ranked list in JSON format, including the product name, average review score,
number of reviews, price, and key review insights for each product. Give a score out of ten to each product. Give the ranking in descending order.
### Instruction ###
Format the ranked list of products in JSON format as follows:
{
"ranked_products": [
{
"product_name": "Product Name",
"average_review_score": "Average Review Score",
"number_of_reviews": "number_of_reviews"
"price": "Price",
"key_review_insights": ["Insight 1", "Insight 2"]
"given_score": "score"
},
...
]
}
Ensure your response adheres to this format, listing products from best to worst as per the criteria outlined.
This instruction is specific, leveraging direct command to format the output in JSON,
and clearly defines the evaluation criteria, focusing on what to do to achieve the desired outcome.
Produce the output in JSON format, containing the product name, average review score,
number of reviews price, given score, and key review insights for each product, without any additional text or explanation.
Analyze the input in a mostly qualitative manner. Do not solely base the ranking and score based off of a python script. Sample Output
{
"ranked_products": [
{
"product_name": "Carhartt Men's Quilted Flannel-Lined Duck Active Jacket, Black",
"average_review_score": "4.7",
"number_of_reviews": "6340",
"price": "$119.99",
"key_review_insights": ["Well made", "Attractive", "Comfortable", "Good wind resistance", "Good water resistance"],
"given_score": "9.5"
},
{
"product_name": "The North Face Men's Hydrenalite Down Hoodie - XXL - TNF Black",
"average_review_score": "4.8",
"number_of_reviews": "540",
"price": "$189.00",
"key_review_insights": ["Attractive", "Well made", "Comfortable", "Good wind resistance", "Good water resistance"],
"given_score": "9"
},
{
"product_name": "Patagonia Isthmus Parka Classic Tan / S",
"average_review_score": "4.6",
"number_of_reviews": "338",
"price": "$187.99",
"key_review_insights": ["Attractive", "Well made", "Good wind resistance", "Good water resistance", "Comfort"],
"given_score": "8.5"
},
{
"product_name": "Obermeyer Men's Foundation Jacket - L - Winter Sky",
"average_review_score": "4.8",
"number_of_reviews": "205",
"price": "$95.96",
"key_review_insights": ["Attractive", "Well made", "Hood functions well", "Comfortable", "Good water resistance"],
"given_score": "8"
},
{
"product_name": "Men's Mountain Classic Down Hooded Jacket, Sherpa-Lined Black Xxxl, Synthetic/Nylon | L.L.Bean, Regular",
"average_review_score": "4.4",
"number_of_reviews": "193",
"price": "$199",
"key_review_insights": ["Attractive", "Well made", "Good wind resistance", "Comfortable", "Water resistance"],
"given_score": "7.5"
}
]
}
LLM Prompt Testing for recommendations and scoring each product
Added pros and Cons
In this issue we test different prompts for recommendations and score each product.
Google folder with inputs and results
In this issue we also test different prompts for ranking pros and cons for each chosen product.
Testing using GPT4 and GPT 3.5