BlinkDL / LM-Trick-Questions

Here we collect trick questions and failed tasks for open source LLMs to improve them.
Apache License 2.0
32 stars 2 forks source link

"Return back this list of words" #9

Open Yuzabotov opened 1 year ago

Yuzabotov commented 1 year ago

Summary

It struggles to return back a list of words. (that look like a python array)

Model version:

RWKV-4-Raven-7B-v10-Eng99%-Other1%-20230418-ctx8192

Example 1

Instruction: return back this list of words Input: [hello,there,how,are,you] Output: [Hello, There, How are you]

Example 2

Instruction: return back this list of words Input: [abc,dsd,aoho,oo] Output: ['a', 'o', 'd', 's']

Example 3

Instruction: return back this list of words Input: [there,are,eleven,cars,in,the,street,2,bicycles,and,a,train,is,also,passing,nearby] Output: There are eleven cars in the street, two bicycles nearby, and a train passing by.

Yuzabotov commented 1 year ago

Re-tested:

RWKV-4-Raven-7B-v11-Eng99%-Other1%-20230427-ctx8192 No noticeable difference