search

BlinkDL / LM-Trick-Questions

Here we collect trick questions and failed tasks for open source LLMs to improve them.
Apache License 2.0
32 stars 2 forks source link

Word replacement #2

Open Yuzabotov opened 1 year ago

Yuzabotov commented 1 year ago

Summary

It fails to return the input sentence with a word replaced

Model version:

RWKV-4-Raven-7B-v10-Eng99%-Other1%-20230418-ctx8192

Example 1

Instruction: return the sentence but replace "how" with "witch" Input: hello there how are you Output: hello witch how are you?

Example 2

Instruction: return the sentence but replace "nice" with "airplane" Input: the weather is nice today Output: The weather is airplane today.

Example 3

Instruction: return the sentence but replace "street" with "orange" Input: there are eleven cars in the street, 2 bicycles and a train is also passing nearby Output: There are eleven cars in the street, two bicycles and a train passing nearby.

Yuzabotov commented 1 year ago

Re-tested:

RWKV-4-Raven-7B-v11-Eng99%-Other1%-20230427-ctx8192 No noticeable difference