duplicated query result

[sgt1796]

The search faiss function sometimes return duplicated functions. But making slight change to user query can significantly change the result. e.g. Asking "Recommand me some spicy chinese food" will give following duplicated results:

sgt1796@cpu-server-instance-1:~$ python3 GPT_embedding/faiss_search_CLI.py --query "Recommand me some spicy chinese food" --db /disk3/GPT_embedding_output/Reviews.db --index /disk3/GPT_embedding_output/IVFPQ_index.bin --top 4

Output:

Query: Recommand me some spicy chinese food

Result 1: Summary: Excellent|Text: This hot chilli sauce tastes truly original. I used to live in Hong Kong and loved to put this sauce on almost everything. And it actually goes with everything, well especially Asian noodle soups and fried rice/noodle. It has the perfect flavor and heat and makes everything tastier. It's not too hot, but just enough to spice everything up. I recommend it to everyone.

Result 2: Summary: Excellent|Text: This hot chilli sauce tastes truly original. I used to live in Hong Kong and loved to put this sauce on almost everything. And it actually goes with everything, well especially Asian noodle soups and fried rice/noodle. It has the perfect flavor and heat and makes everything tastier. It's not too hot, but just enough to spice everything up. I recommend it to everyone.

Result 3: Summary: Excellent|Text: This hot chilli sauce tastes truly original. I used to live in Hong Kong and loved to put this sauce on almost everything. And it actually goes with everything, well especially Asian noodle soups and fried rice/noodle. It has the perfect flavor and heat and makes everything tastier. It's not too hot, but just enough to spice everything up. I recommend it to everyone.

Result 4: Summary: Good to spike up your soup|Text: I like to put a few drops of this to my soup, to spike it up a little bit just enough to tickle my appetite.  It is made up of soybean oil, sesame oil, chili peppers, and some other ingredients.  There are actually pieces of peppers and seeds 1/4 at the bottom of the bottle.<br /><br />Nice dip for dimsums, chow meins, or other food that you want add some extra kick.  I like the slightly different taste of this oil based chili sauce when I get tired of the taste of water based Tabasco or Sriracha.  Same oil-based chili dip you see on the table of most Chinese restaurants.

But adding a newline character "\n" after the query, i.e. "Recommand me some spicy chinese food\n" will result the followin:

sgt1796@cpu-server-instance-1:~$ python3 GPT_embedding/faiss_search_CLI.py --query "Recommand me some spicy chinese food\n" --db /disk3/GPT_embedding_outp
ut/Reviews.db --index /disk3/GPT_embedding_output/IVFPQ_index.bin --top 4

Output:

Query: Recommand me some spicy chinese food\n

Result 1: Summary: Best spicy noodle on the market|Text: Although this noodle is the "default" according to my Korean friend, I believe it is the best spicy noodle on the market. It is VERY spicy for those that are not accustomed to spicy food. If you have not tried this noodle before, I'd recommend using only half the chili powder packet. Then adjust as necessary for later meals. I've been eating this noodle for years and years. The recipe that I normally use is:<br /><br />1 egg<br />Chinese green onions (chopped)<br />lemon juice<br />sesame oil<br /><br />Bring water to a boil with the noodle in a pot. Then pour out the water and run cold water over the noodle. This cleans the noodles of the materials left over from the manufacturing process. Set noodle aside. Then bring filtered water to a boil with both ingredient packets. It takes several minutes for the dried vegetables to hydrate. Put noodles back in. Add 3-4 drops of lemon juice and 1-2 drops of sesame oil. Add the green onions. Stir. Break the egg over the top of the noodles. Make sure your hands are dry to avoid steam burns! Wait until the egg white turns from translucent to white. Serve over large bowl. Break the egg and allow the yolk to run over the noodles. YUM!

Result 2: Summary: Very good Spicy Noodle|Text: I really like spicy noodle dishes, and this one is great.  It's a Korean noodle dish, so the spice is inspired by the flavor of kimchi, which I love, so this was really tasty for me.<br /><br />It's pretty spicy, but not insanely spicy.  Spicy enough so that your friends who don't like spice won't steal it, but you've probably had spicier foods than this.

Result 3: Summary: Chinese comfort food|Text: I'm not Chinese (in this life anyway), but I've found two Chinese comfort foods that make me just as happy as mac and cheese and P, B & J on soft white bread with cold milk.  My two faves are:  1.  Mabo (or Mapo) tofu with rice, and 2.  Chinese sausage (lop chong?) cut up and steamed with rice and served with a dash of soy sauce, sesame oil and chopped scallions.  Yummy!

Result 4: Summary: yummy spicy, somewhat sweet and tasty ramen noodles.|Text: The BBQ Chicken Mi Goreng flavor packs on the heat...and just right for those who like their food on the intense side. It has lots of packets of flavor including a spicy chili oil and soy sauce and some msg-powdered seasoning and hot chili powder and even dehydrated fried onions as garnish.<br /><br />It is subtly sweet and the spicy-heat just hits you in every bite and the flavors all merge into deliciousness. This is a flavor unlike any asian bistro cooking, and yet, it's so-oooooooooooo so, so good.<br /><br />I must admit that I have often wanted to find other flavors that were just as tasty and spicy and have only found 1 that is nearly as tasty but it's not a similar sweet-n-hot-savory, it's got heat but it's very different. (see my other ramen noodle review)