search

BlinkDL / LM-Trick-Questions

Here we collect trick questions and failed tasks for open source LLMs to improve them.
Apache License 2.0
32 stars 2 forks source link

"Classify each line as either a Statement or a Question." #7

Open Yuzabotov opened 1 year ago

Yuzabotov commented 1 year ago

Summary

It struggles classifying a list of sentences, all given at once, as either statements or questions

Model version:

RWKV-4-Raven-7B-v10-Eng99%-Other1%-20230418-ctx8192

Example 1

Instruction: Classify each of the following lines of text as either a Statement or a Question. Input: 1 How can I help you? 2 I was wondering about my application. 3 I have a class. 4 Where do I purchase textbooks? 5 How do i add classes? Output: 1. Statement 2. Question 3. Statement 4. Question 5. Statement

Example 2

Instruction: Classify each of the following lines of text as either a Statement or a Question. Input: 1 Bricks are a building material. 2 He asked "how are you doing?" 3 I remember. 4 It's one of the best. 5 What is it? Output: 1. Statement 2. Question 3. Statement 4. Statement 5. Statement

Example 3

Instruction: Classify each of the following lines of text as either a Statement or a Question. Input: 1 I like bananas. 2 I love refrigerators. 3 I have a giant plate of spaghetti? 4 There are eleven cars in the street, 2 bicycles and a train is also passing nearby. 5 Hi? Output: Statement: 1, 2, 3, 4 Question: 5

Yuzabotov commented 1 year ago

Re-tested:

RWKV-4-Raven-7B-v11-Eng99%-Other1%-20230427-ctx8192 No noticeable difference