multiple WARNING list 'user.XXX' skipped unknown tokens

[saidelike]

With latest commit (11e09f8fc481f11c31ee95effdffbc748aa6d338) and Talon 0.3.1-630

With only knausj_talon in Talon user, I have these warnings when starting Talon or after saying a couple of commands like say hello or dictation mode and saying anything:

2023-06-14 13:00:20 WARNING list 'user.abbreviation' skipped unknown tokens: ['-', '.', '3', '5', ':', '[', ']']
2023-06-14 13:00:20 WARNING list 'user.punctuation' skipped unknown tokens: [',', '`']
2023-06-14 13:00:20 WARNING list 'user.symbol_key' skipped unknown tokens: [',', '`']
2023-06-14 13:02:07 WARNING skipped unknown tokens: ['-']

Seems similar to https://github.com/knausj85/knausj_talon/issues/440 but not directly linked.

It seems related to contents like core/abbreviate.py:

from talon import Context, Module

from ..user_settings import get_list_from_csv

mod = Module()
mod.list("abbreviation", desc="Common abbreviation")

abbreviations = {
    "J peg": "jpg",
    "abbreviate": "abbr",
...
    "jason five": "json5",
...

which generates settings/abbreviations.csv containing character "5", etc.

If I remove that json5 entry from my CSV, the warning has one less character (the character 5):

2023-06-14 13:13:18 WARNING list 'user.abbreviation' skipped unknown tokens: ['-', '.', '3', ':', '[', ']']

[nriley]

I figured out what was causing at least the abbreviation issue tonight - the code here.

This is taking an abbreviation expansion (or contraction, such that it is...) and trying to map it as a speech command, so you can say "brief" followed by either the expanded or abbreviated version of an abbreviation. This is invalid; we should either see about skipping these ourselves, or decide if there is some way to map it sensibly (create_spoken_forms as is would not be suitable, I think).

[AndreasArvidsson]

I agree. Create spoken forms is probably the best solution here.