search

BlinkDL / LM-Trick-Questions

Here we collect trick questions and failed tasks for open source LLMs to improve them.
Apache License 2.0
32 stars 2 forks source link

Word counting #10

Open Yuzabotov opened 1 year ago

Yuzabotov commented 1 year ago

Summary

It struggles counting the number of words in a sentence

Model version:

RWKV-4-Raven-7B-v10-Eng99%-Other1%-20230418-ctx8192

Example 1

Instruction: Count the number of words in the sentence Input: hello there how are you Output: The sentence "hello there how are you" has 3 words.

Example 2

Instruction: Count the number of words in the sentence Input: Count the number of words in the sentence Output: The sentence has 10 words.

Example 3

Instruction: Count the number of words in the sentence Input: what is three plus four? Output: The sentence "what is three plus four?" has two words: "what" and "is".

Yuzabotov commented 1 year ago

Re-tested:

RWKV-4-Raven-7B-v11-Eng99%-Other1%-20230427-ctx8192 No noticeable difference