instructlab / taxonomy

Taxonomy tree that will allow you to create models tuned with your data
Apache License 2.0
177 stars 614 forks source link

Scramble words output is not consistent. #504

Closed amtdas closed 6 months ago

amtdas commented 6 months ago

Describe the bug/problem

Input given at the prompt

>>> Unscramble words with  TNO?                                                                                                                  [S][default]
╭─────────────────────────────────────────────────────────────────────── assistant ────────────────────────────────────────────────────────────────────────╮
│ Here are some words that can be unscrambled from the letters in "TNO":                                                                                   │
│                                                                                                                                                          │
│ * NOT (TON -> NOT)                                                                                                                                       │
│ * TON (NOT -> TON)                                                                                                                                       │
│ * ONTO (TON -> ONTO)                                                                                                                                     │
│ * TONY (TON -> TONY)                                                                                                                                     │
│ * NOTY (TON -> NOTY)                                                                                                                                     │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 2.908 seconds ─╯
>>> Unscramble words with NTO?                                                                                                                  [S][default]
╭────────────────────────────────────────────────────────────────── merlinite-7b-Q4_K_M ───────────────────────────────────────────────────────────────────╮
│ Certainly! I can help you unscramble words containing the letters N, T, and O. Here are some possible results:                                           │
│                                                                                                                                                          │
│ 1. Not too bad: This phrase is often used to express that something is not terrible, but could still be improved.                                        │
│ 2. Tent on: This phrase is not commonly used, but it could potentially mean "tenting on" or "focusing on" something.                                     │
│ 3. Tonot: This is not a standard English word, but it could be interpreted as "not a tone" or "not a sound."                                             │
│ 4. Tonto: This is a name, often associated with the character from the Lone Ranger series, meaning "only" or "alone" in Spanish.                         │
│ 5. Tortonian: This term is used in geology to refer to the Tortonian stage of the Pliocene epoch, which lasted from about 4.8 to 1.8 million years ago.  │
│ 6. Tonto's: This is not a standard English word, but it could be interpreted as "Tonto's" something, referring to the character from the Lone Ranger     │
│ series possessing something.                                                                                                                             │
│ 7. Tonto's: This is not a standard English word, but it could be interpreted as "Tonto's" something, referring to the character from the Lone Ranger     │
│ series possessing something.                                                                                                                             │
│ 8. Tonto's: This is not a standard English word, but it could be interpreted as "Tonto's" something, referring to the character from the Lone Ranger     │
│ series possessing something.                                                                                                                             │
│ 9. Tonto's: This is not a standard English word, but it could be interpreted as "Tonto's" something, referring to the character from the Lone Ranger     │
│ series possessing something.                                                                                                                             │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 12.342 seconds ─╯

Response that was received from the current model

I have checked several other words and seems lab chat output is different. Output words are lesser. It should return 2 letters words, 3 letter words and 4 letter words. Here, ACT, EAT , ATE are missing along with others.

>>> Unscramble words with TCAE?                                                                                                                 [S][default]
╭─────────────────────────────────────────────────────────────────────── assistant ────────────────────────────────────────────────────────────────────────╮
│ Here are some words that can be unscrambled from the letters in "TCAE":                                                                                  │
│                                                                                                                                                          │
│ * CAT (CATE -> CAT)                                                                                                                                      │
│ * ECT (TEAC -> TEAC)                                                                                                                                     │
│ * ACE (ETAC -> ACE)                                                                                                                                      │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 2.206 seconds ─╯

Response that you expected instead

All the respective dictionary words with these . For example:

>>> Unscramble words with TNO? 

NOT 
TON
ON
NO

Please let us know if this is a correction or new proposal.

obuzek commented 6 months ago

It's worth a try with a skill like this as it's something LLMs might technically be able to do, but they're likely to struggle, as you see here.

I think I've seen a PR or two trying to implement this behavior, so before submitting a new one, please check to see if there are other folks you could collaborate with to make a more effective version of the skill.

amtdas commented 5 months ago

Hello - Thanks for information. How about creating a discussion forum and collaborating with others? Kindly let me know if it can be initiated. Thanks