jwebmeister / tacspeak

Tacspeak - Fast, lightweight, modular speech recognition for gaming
GNU Affero General Public License v3.0
42 stars 2 forks source link

Nothing happens in game #30

Closed ShadowESH123 closed 3 months ago

ShadowESH123 commented 3 months ago

I have everthing setup but nothing happens in game. Could it be because I use azerty ? I change the keybindings and run it as admin but it doesnt work. This is what I get in Tacspeak but nothing happens in game:

Please help

Tacspeak version 0.2.0

    Tacspeak - speech recognition for gaming
    © Copyright 2023-2024 by Joshua Webb

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU Affero General Public License as
    published by the Free Software Foundation, either version 3 of the
    License, or (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU Affero General Public License for more details.

    You should have received a copy of the GNU Affero General Public License
    along with this program.  If not, see <https://www.gnu.org/licenses/>.

engine (INFO): Initialized 'kaldi' SR engine: KaldiEngine().
engine (INFO): Loading Kaldi-Active-Grammar v3.1.0 in process 16976.
engine (INFO): Kaldi options: {'model_dir': None, 'tmp_dir': None, 'audio_input_device': None, 'audio_self_threaded': True, 'audio_auto_reconnect': True, 'audio_reconnect_callback': None, 'retain_dir': None, 'retain_audio': False, 'retain_metadata': False, 'retain_approval_func': None, 'vad_aggressiveness': 3, 'vad_padding_start_ms': 150, 'vad_padding_end_ms': 150, 'vad_complex_padding_end_ms': 600, 'auto_add_to_user_lexicon': False, 'allow_online_pronunciations': False, 'lazy_compilation': True, 'invalidate_cache': False, 'expected_error_rate_threshold': None, 'alternative_dictation': None, 'compiler_init_config': {}, 'decoder_init_config': {}, 'listen_key': 226, 'listen_key_toggle': 2, 'listen_key_padding_end_ms_min': 1, 'listen_key_padding_end_ms_max': 170, 'listen_key_padding_end_always_max': False}
engine (INFO): streaming audio from 'Microfoonmatrix (Realtek(R) Aud' using MME: 16000 sample_rate, 10 block_duration_ms, 30 latency_ms
directory (INFO): Looking for command modules here: C:\Program Files\Tacspeak-3159-0-2-0-1705910527\tacspeak\grammar
directory (INFO): Valid paths: C:\Program Files\Tacspeak-3159-0-2-0-1705910527\tacspeak\grammar\_readyornot.py
module (INFO): CommandModule('_readyornot.py'): Loading module: 'C:\Program Files\Tacspeak-3159-0-2-0-1705910527\tacspeak\grammar\_readyornot.py'
-- Ready or Not keybindings --
gold:'debug_print_key'()
blue:'debug_print_key'()
red:'debug_print_key'()
alpha:'debug_print_key'()
bravo:'debug_print_key'()
charlie:'debug_print_key'()
delta:'debug_print_key'()
cmd_1:'debug_print_key'()
cmd_2:'debug_print_key'()
cmd_3:'debug_print_key'()
cmd_4:'debug_print_key'()
cmd_5:'debug_print_key'()
cmd_6:'debug_print_key'()
cmd_7:'debug_print_key'()
cmd_8:'debug_print_key'()
cmd_9:'debug_print_key'()
cmd_back:'debug_print_key'()
cmd_hold:'debug_print_key'()
cmd_default:'debug_print_key'()
cmd_menu:'debug_print_key'()
interact:'debug_print_key'()
yell:'debug_print_key'()
-- Ready or Not keybindings --
engine (INFO): Loading grammar ReadyOrNot
engine (INFO): Loading grammar ReadyOrNot_priority
engine (INFO): Loading grammar _recobs_grammar
Ready to listen...
engine (INFO): Listening...
engine (INFO): Cold mic
engine (INFO): Hot mic
kaldi.wrapper (Level 15): decoded at 0.13 RTF, for 670 ms audio, spending 87 ms, of which 53 ms (60%) in finalization
on_recognition (INFO): KaldiRule(7, ReadyOrNot::StackUp) | team stack up right
on_recognition (INFO): KaldiRule(7, ReadyOrNot::StackUp) | team stack up right
on_recognition (INFO): KaldiRule(7, ReadyOrNot::StackUp) | team stack up right
on_recognition (INFO): KaldiRule(7, ReadyOrNot::StackUp) | team stack up right
current team go stack up right
action.exec (DEBUG): Executing action: 'NULL_ACTION = Function(lambda: print("NULL_ACTION")\n                       if DEBUG_NOCMD_PRINT_ONLY else None)'()+'debug_print_key'()+'debug_print_key'()+'debug_print_key'() (None)
action.exec (DEBUG): Executing action: 'NULL_ACTION = Function(lambda: print("NULL_ACTION")\n                       if DEBUG_NOCMD_PRINT_ONLY else None)'() (None)
NULL_ACTION
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(m_middle)
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(kb_1)
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(kb_3)
engine (Level 15): End of phrase: eer=0.00 conf=nan, rule KaldiRule(7, ReadyOrNot::StackUp), 'team stack up right'
kaldi.wrapper (Level 15): decoded at 0.13 RTF, for 920 ms audio, spending 119 ms, of which 64 ms (54%) in finalization
on_recognition (INFO): KaldiRule(11, ReadyOrNot::FallIn) | on me
on_recognition (INFO): KaldiRule(11, ReadyOrNot::FallIn) | on me
on_recognition (INFO): KaldiRule(11, ReadyOrNot::FallIn) | on me
on_recognition (INFO): KaldiRule(11, ReadyOrNot::FallIn) | on me
current team go fall in single
action.exec (DEBUG): Executing action: 'NULL_ACTION = Function(lambda: print("NULL_ACTION")\n                       if DEBUG_NOCMD_PRINT_ONLY else None)'()+'debug_print_key'()+'debug_print_key'()+'debug_print_key'() (None)
action.exec (DEBUG): Executing action: 'NULL_ACTION = Function(lambda: print("NULL_ACTION")\n                       if DEBUG_NOCMD_PRINT_ONLY else None)'() (None)
NULL_ACTION
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(m_middle)
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(kb_2)
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(kb_1)
kaldi.wrapper (Level 15): decoded at 0.10 RTF, for 830 ms audio, spending 85 ms, of which 38 ms (45%) in finalization
on_recognition (INFO): KaldiRule(10, ReadyOrNot::GroundOptions) | stop
on_recognition (INFO): KaldiRule(10, ReadyOrNot::GroundOptions) | stop
on_recognition (INFO): KaldiRule(10, ReadyOrNot::GroundOptions) | stop
on_recognition (INFO): KaldiRule(10, ReadyOrNot::GroundOptions) | stop
current team go halt
action.exec (DEBUG): Executing action: 'NULL_ACTION = Function(lambda: print("NULL_ACTION")\n                       if DEBUG_NOCMD_PRINT_ONLY else None)'()+'debug_print_key'()+'debug_print_key'() (None)
action.exec (DEBUG): Executing action: 'NULL_ACTION = Function(lambda: print("NULL_ACTION")\n                       if DEBUG_NOCMD_PRINT_ONLY else None)'() (None)
NULL_ACTION
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(m_middle)
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(kb_4 )
kaldi.wrapper (Level 15): decoded at 0.11 RTF, for 1110 ms audio, spending 120 ms, of which 53 ms (45%) in finalization
on_recognition (INFO): KaldiRule(11, ReadyOrNot::FallIn) | fall in
on_recognition (INFO): KaldiRule(11, ReadyOrNot::FallIn) | fall in
on_recognition (INFO): KaldiRule(11, ReadyOrNot::FallIn) | fall in
on_recognition (INFO): KaldiRule(11, ReadyOrNot::FallIn) | fall in
current team go fall in single
action.exec (DEBUG): Executing action: 'NULL_ACTION = Function(lambda: print("NULL_ACTION")\n                       if DEBUG_NOCMD_PRINT_ONLY else None)'()+'debug_print_key'()+'debug_print_key'()+'debug_print_key'() (None)
action.exec (DEBUG): Executing action: 'NULL_ACTION = Function(lambda: print("NULL_ACTION")\n                       if DEBUG_NOCMD_PRINT_ONLY else None)'() (None)
NULL_ACTION
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(m_middle)
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(kb_2)
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(kb_1)
kaldi.wrapper (Level 15): decoded at 0.10 RTF, for 1520 ms audio, spending 153 ms, of which 44 ms (29%) in finalization
on_recognition (INFO): KaldiRule(7, ReadyOrNot::StackUp) | gold team stack up
on_recognition (INFO): KaldiRule(7, ReadyOrNot::StackUp) | gold team stack up
on_recognition (INFO): KaldiRule(7, ReadyOrNot::StackUp) | gold team stack up
on_recognition (INFO): KaldiRule(7, ReadyOrNot::StackUp) | gold team stack up
gold team go stack up split
action.exec (DEBUG): Executing action: 'debug_print_key'()+'debug_print_key'()+'debug_print_key'()+'debug_print_key'() (None)
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(kb_f5)
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(m_middle)
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(kb_1)
action.exec (DEBUG): Executing action: 'debug_print_key'() (None)
(kb_1)
engine (INFO): Cold mic
jwebmeister commented 3 months ago

Check your user_settings.py, specifically it should contain DEBUG_MODE = False.

If DEBUG_MODE is set to True, Tacspeak will print the command and debug text but not press any virtual keys, basically doing nothing in-game.

ShadowESH123 commented 3 months ago

my setttings

DEBUG_MODE = False, # enables additional logging, and if properly setup in the grammar module:

- enables module without needing the app in focus, i.e. AppContext()

                                            # - actions only print to console, they don't press virtual keys

DEBUG_HEAVY_DUMP_GRAMMAR = False, # expensive on memory, don't set this to True unless you're sure

if properly setup in the grammar module:

                                            # - generates a .debug_grammar_*.txt that describes the spec of the active commands

USE_NOISE_SINK = True, # load NoiseSink rule(s), if it's setup in the grammar module.

- it should partially capture other noises and words outside of commands, and do nothing.

def my_retain_func(audio_store): """Used in retain_approval_func""" return not 'NoiseSink' in audio_store.rule_name

KALDI_ENGINE_SETTINGS = { "listen_key":0xE2, # see https://learn.microsoft.com/en-us/windows/win32/inputdev/virtual-key-codes

0x10=SHIFT key

                                            # 0xE2=Next to SHIFT key
                                            # 0x05=X1 mouse button
                                            # 0x06=X2 mouse button
                                            # None=overrides listen_key_toggle, and sets it into always listening mode; 
                                            #      uses Voice Activity Detector (VAD) to detect end of speech and recognise commands.
"listen_key_toggle":2,                      # Recommended is 0 or -1. 
                                            #  0 for toggle mode off; 
                                            #  1 for toggle mode on; 
                                            #  2 for global toggle mode on (use VAD); 
                                            #  -1 for toggle mode off but allow priority grammar even when key not pressed
"listen_key_padding_end_ms_min":1,          # min ms of audio captured after listen_key is released (or toggled off), after which if VAD detects silence it will stop capturing; 
                                            # recommended is 1 if using listen_key_toggle (0 or -1), 0 for anything else.
"listen_key_padding_end_ms_max":170,        # max ms of audio captured after listen_key is released (or toggled off), but will stop short if VAD detects silence; 
                                            # recommended is 170 if using listen_key_toggle (0 or -1), 0 for anything else.
"listen_key_padding_end_always_max":False,  # disregard VAD and always capture listen_key_padding_end_ms_max of audio after listen_key is released (or toggled off)
"vad_padding_end_ms":150,                   # ms of required silence after VAD; 
                                            # recommended is 150 if using listen_key_toggle (0 or -1), 250 for anything else.
"auto_add_to_user_lexicon":False,           # this requires g2p_en (which isn't installed by default)
"allow_online_pronunciations":False,
# "retain_dir":"./retain/",                 # uncomment this to retain recordings of recognised commands. set to a writeable directory to retain recognition metadata and/or audio data
# "retain_audio":True,                      # uncomment this to retain recordings of recognised commands. set to True to retain audio of recognitions in .wav files in the retain_dir (if set)
# "retain_metadata":True,                   # uncomment this to retain recordings of recognised commands. set to True to retain metadata of recognitions in a .tsv file in retain_dir (if set)
# "retain_approval_func":my_retain_func,    # uncomment this to retain recordings of recognised commands. set to a function returning `True` or `False` based on `AudioStoreEntry` contents
# "audio_input_device":None,                # set to an int to choose a non-default microphone. use "./tacspeak.exe --print_mic_list" to see what devices are available.
# "input_device_index":None,
# "vad_aggressiveness":3,                   # default aggressiveness of VAD
# "vad_padding_start_ms":150,               # default ms of required silence before VAD
# "model_dir":'kaldi_model',                # default model directory
# "tmp_dir":None, 
# "audio_self_threaded":True, 
# "audio_auto_reconnect":True, 
# "audio_reconnect_callback":None,
# "vad_complex_padding_end_ms":600,         # default ms of required silence after VAD for complex utterances
# "lazy_compilation":True,                  # set to True to parallelize & speed up loading
# "invalidate_cache":False,
# "expected_error_rate_threshold":None,
# "alternative_dictation":None,
# "compiler_init_config":None, 
# "decoder_init_config":None,

}

ShadowESH123 commented 3 months ago

my read or not file:

#

This file is part of Tacspeak.

(c) Copyright 2023 by Joshua Webb

Licensed under the AGPL-3.0; see LICENSE.txt file.

#

import sys import dragonfly from dragonfly import (BasicRule, CompoundRule, MappingRule, RuleRef, Repetition, RecognitionObserver, Function, Choice, IntegerRef, Grammar, Alternative, Literal, Text, Optional, AppContext, Dictation)

from dragonfly.engines.backend_kaldi.dictation import UserDictation as Dictation

from dragonfly.actions import (Key, Mouse, ActionBase)

from kaldi_active_grammar import KaldiRule

---------------------------------------------------------------------------

Check DEBUG_MODE (from user_settings)

try: DEBUG_MODE = (sys.modules["user_settings"]).DEBUG_MODE except Exception: DEBUG_MODE = False

try: DEBUG_HEAVY_DUMP_GRAMMAR = (sys.modules["user_settings"]).DEBUG_HEAVY_DUMP_GRAMMAR except Exception: DEBUG_HEAVY_DUMP_GRAMMAR = False

whether or not to load NoiseSink rule to grammar_priority

try: USE_NOISE_SINK = (sys.modules["user_settings"]).USE_NOISE_SINK except Exception: USE_NOISE_SINK = False

DEBUG_MODE = True # if you want to override

DEBUG_HEAVY_DUMP_GRAMMAR = True # if you want to override

USE_NOISE_SINK = False # if you want to override

---------------------------------------------------------------------------

Create this module's grammar and the context under which it'll be active.

if DEBUG_MODE: grammar_context = AppContext() else: grammar_context = AppContext(executable="ReadyOrNot") grammar = Grammar("ReadyOrNot", context=grammar_context, ) grammar_priority = Grammar("ReadyOrNot_priority", context=grammar_context, )

---------------------------------------------------------------------------

Variables used by grammar, rules, recognition observers below

Users should be able to look here first for customisation

Will map keybindings to print()

DEBUG_NOCMD_PRINT_ONLY = DEBUG_MODE

the minimum time between keys state changes (e.g. pressed then released),

it's to make sure key presses are registered in-game

min_delay = 3.3 # 100/(30 fps) = 3.3 (/100 seconds between frames)

map of action to in-game key bindings

https://dragonfly.readthedocs.io/en/latest/actions.html#key-names

https://dragonfly.readthedocs.io/en/latest/actions.html#mouse-specification-format

ingame_key_bindings = { "gold": "f5", "blue": "f6", "red": "f7", "alpha": "f13", "bravo": "f14", "charlie": "f15", "delta": "f16", "cmd_1": "ampersand", "cmd_2": "é", "cmd_3": "quote", "cmd_4": "apostrophe ", "cmd_5": "leftparen", "cmd_6": "§", "cmd_7": "è", "cmd_8": "exclamation", "cmd_9": "ç", "cmd_back": "tab", "cmd_hold": "comma", "cmd_default": "z", "cmd_menu": "mouse_middle", "interact": "f", "yell": "f", }

def debug_printkey(device, key): print(f'({device}{key})')

if DEBUG_NOCMD_PRINT_ONLY: map_ingame_key_bindings = {k: Function(debug_printkey, device='m', key=v.replace("mouse", "")) if "mouse_" in v else Function(debug_print_key, device='kb', key=v) for k, v in ingame_key_bindings.items()} else: map_ingame_keybindings = {k: Mouse(f'{v.replace("mouse", "")}:down/{mindelay}, {v.replace("mouse", "")}:up') if "mouse_" in v else Key(f'{v}:down/{min_delay}, {v}:up') for k, v in ingame_key_bindings.items()}

print key bindings

print("-- Ready or Not keybindings --") for (k, v) in map_ingame_key_bindings.items(): print(f'{k}:{v}') print("-- Ready or Not keybindings --")

mappings of spoken phrases to values

map_hold = { "on my (mark | order | command)": "hold", } map_execute_or_cancels = { "execute": "execute", "cancel": "cancel", "go [go go]": "execute", } map_colors = { "gold": "gold", "blue": "blue", "red": "red", } map_door_options = {

note: stackup, breach & clear, open & clear, scan are separate options

"mirror [under]": "mirror",
"wand [under]": "mirror",
"disarm": "disarm", # todo! this inserts itself in the middle of the list and messes up other keybinds, update when Void updates
"wedge": "wedge", # "remove the wedge" has its own recognition. can specify if door is "trapped door" to use correct keybinds.
"block": "wedge",
"cover": "cover", # can specify if door is "trapped door" to use correct keybinds.
"open": "open", # can specify if door is "trapped door" to use correct keybinds.
"close": "close", # can specify if door is "trapped door" to use correct keybinds.

} map_door_trapped = { "trapped": "trapped", } map_door_stack_sides = {

todo! in 1.0 some doors don't have all stack options available, update when Void updates

"split": "split",
"left": "left",
"right": "right",
"auto": "auto",  

} map_door_breach_tools = { "open": "open", "move in": "open", "kick [it] [down]": "kick", "kick the door down": "kick", "(shotgun | shot e)": "shotgun", "c two": "c2", "[battering] ram [it]": "ram", "((leader | lead) will | wait for (my | me to)) (open | breach)": "leader", } map_door_grenades = { "(bang | flash bang | flash)": "flashbang", "stinger": "stinger", "(cs | gas | cs gas)": "gas", "[the] (fourty mil | launcher)": "launcher", "((leader | lead) will | wait for (my | me to)) (grenade | flash bang | bang | flash | stinger | cs | gas | cs gas | fourty mil | launcher)": "leader", } map_door_scan = {

todo! in 1.0 some doors don't have all scan options available, update when Void updates

"scan": "pie",
"slide": "slide",
"pie": "pie",
"peek": "peek",

} map_ground_options = {

note: deploy and fall in are separate options

"move ([over] (here | there) | [to] that (location | position))": "move",
"cover ([over] (here | there) | that (location | position))": "cover",
"(hold | halt | stop) [(position | movement)]": "halt",
"resume [movement]": "resume",
"(secure | search) [the] (area | room)": "search",
"(search for | collect | secure) evidence": "search",

} map_ground_fallin_formations = { "single file": "single", "double file": "double", "diamond [formation]": "diamond", "wedge [formation]": "wedge", } map_ground_deployables = { "(bang | flash bang | flash)": "flashbang", "stinger": "stinger", "(cs | gas | cs gas)": "gas", "chem light": "chemlight", "shield": "shield", } map_npc_player_interacts = { "move [(here | there)]": "move here", "(move | come) to (me | my position)": "move my position", "come here": "move my position", "stop [(there | moving | movement)]": "move stop", "turn around": "turn around", "move to [the] exit": "move to exit", "get out of here": "move to exit", "(get | move) outside": "move to exit", } map_npc_team_restrain = { "restrain": "restrain", "arrest": "restrain", } map_npc_team_deployables = { "taser": "taser", "tase": "taser", "pepper spray": "pepperspray", "pepper ball": "pepperball", "bean [bag]": "beanbag", "melee": "melee", "violence": "melee", } map_team_members = { "alpha": "alpha", "bravo": "bravo", "charlie": "charlie", "delta": "delta", } map_team_member_options = { "move": "move", "(focus | watch)": "focus", "(un | release) (focus | watch)" : "unfocus", "stop (focusing | watching)" : "unfocus", "swap with": "swap", "(search | secure) [the] (room | area)": "search", } map_team_member_move = { "([over] (here | there) | [to] that (location | position))": "here", "[over] ((here | there) | [to] that (location | position)) then back": "here then back", } map_team_member_focus = { "([over] (here | there) | [on] that (location | position))": "here", "([on] my position | on me)": "my position", "[on] [(the | that)] door [way]": "door", "[on] (em | them | him | her | [the] target)": "target", "(un focus | release)": "unfocus", }

def invert_squash_map(my_map): """ Returns an inverted map, where keys are the original values, and values are the concatenation of the original keys as alternative choices. For example: {'a':1,'b':1,'c':1} => {1: '(a | b | c)'} """ inv_map = {} for k, v in my_map.items(): inv_map[v] = inv_map.get(v, []) + [k] for k, v in inv_map.items(): inv_map[k] = '(' + ' | '.join('(' + v + ')') + ')' if len(v) > 1 else ''.join(v) return inv_map

---------------------------------------------------------------------------

Rules which will be added to our grammar

used to chain actions together, e.g. (NULL_ACTION + Key(...) + Mouse(...)).execute()

NULL_ACTION = Function(lambda: print("NULL_ACTION") if DEBUG_NOCMD_PRINT_ONLY else None)

def action_hold(direction): """ press "down" or release "up" the hold command key (on execution)

------------------------------------------------------------------

def cmd_execute_or_cancel_held_order(color, execute_or_cancel): """ Press & release command keys for team to execute held order """ actions = cmd_select_team(color) actions += map_ingame_key_bindings["cmd_menu"] match execute_or_cancel: case "execute": actions += map_ingame_key_bindings["cmd_1"] case "cancel": actions += map_ingame_key_bindings["cmd_2"] return actions

class ExecuteOrCancelHeldOrder(CompoundRule): """ Speech recognise team execute or cancel a held order """ spec = " [team] [([that] [held] order | that [order])]" extras = [ Optional(Choice("color_choice", map_colors), "color", "current"), Choice("execute_or_cancel", map_execute_or_cancels), ] defaults = { "color": "current", "execute_or_cancel": "execute", }

def _process_recognition(self, node, extras):
    color = extras["color"]
    execute_or_cancel = extras["execute_or_cancel"]
    print(f"{color} team {execute_or_cancel} held order")
    cmd_execute_or_cancel_held_order(color, execute_or_cancel).execute()

------------------------------------------------------------------

def cmd_select_team(color): """ Press & release select color team key (on execution), or return NULL_ACTION """ if color != "current": return map_ingame_key_bindings[color] else: return NULL_ACTION

class SelectTeam(CompoundRule): """ Speech recognise select color team """ spec = " team" extras = [Optional(Choice("color_choice", map_colors), "color", "current")] defaults = {"color": "current"}

def _process_recognition(self, node, extras):
    color = extras["color"]
    print(f"Select {color}")
    cmd_select_team(color).execute()

class SelectColor(CompoundRule): """ Speech recognise select color team """ spec = "" extras = [Choice("color", ["blue", "red", "gold"])]

def _process_recognition(self, node, extras):
    color = extras["color"]
    print(f"Select {color}")
    cmd_select_team(color).execute()

------------------------------------------------------------------

def cmd_door_options(color, hold, door_option, trapped): """ Press & release command keys for team to mirror under, wedge, cover, open,

close the door (on execution)

"""
actions = cmd_select_team(color)
actions += map_ingame_key_bindings["cmd_menu"]
# start hold for command
if hold == "hold":
    actions += action_hold("down")
match door_option:
    case "slide":
        actions += map_ingame_key_bindings["cmd_4"]
        actions += map_ingame_key_bindings["cmd_1"]
    case "pie":
        actions += map_ingame_key_bindings["cmd_4"]
        actions += map_ingame_key_bindings["cmd_2"]
    case "peek":
        actions += map_ingame_key_bindings["cmd_4"]
        actions += map_ingame_key_bindings["cmd_3"]
    case "mirror":
        actions += map_ingame_key_bindings["cmd_5"]
    case "disarm":
        actions += map_ingame_key_bindings["cmd_6"]
    case "wedge":
        if trapped == "trapped":
            actions += map_ingame_key_bindings["cmd_7"]
        else:
            actions += map_ingame_key_bindings["cmd_6"]
    case "cover":
        if trapped == "trapped":
            actions += map_ingame_key_bindings["cmd_8"]
        else:
            actions += map_ingame_key_bindings["cmd_7"]
    case "open":
        if trapped == "trapped":
            actions += map_ingame_key_bindings["cmd_9"]
        else:
            actions += map_ingame_key_bindings["cmd_8"]
    case "close":
        if trapped == "trapped":
            actions += map_ingame_key_bindings["cmd_9"]
        else:
            actions += map_ingame_key_bindings["cmd_8"]
# end hold for command
if hold == "hold":
    actions += action_hold("up")
return actions

class DoorOptions(CompoundRule): """ Speech recognise team mirror under, wedge, cover, open, close the door """ spec = " [team] [(the | that)] (door [way] | opening | room)" extras = [ Optional(Choice("color_choice", map_colors), "color", "current"), Optional(Choice("hold_choice", map_hold), "hold", "go"), Choice("door_option", map_door_options | map_door_scan), Optional(Choice("trapped_choice", map_door_trapped), "trapped", "not trapped"), ] defaults = { "color": "current", "hold": "go", "door_option": "open", "trapped": "not trapped", }

def _process_recognition(self, node, extras):
    color = extras["color"]
    hold = extras["hold"]
    door_option = extras["door_option"]
    trapped = extras["trapped"]
    print(f"{color} team {hold} {door_option} {trapped} the door")
    cmd_door_options(color, hold, door_option, trapped).execute()

class WedgeIt(CompoundRule): """ Speech recognise team wedge it """ spec = " [team] (wedge | block) it [(the | that)] [door] [way]" extras = [ Optional(Choice("color_choice", map_colors), "color", "current"), Optional(Choice("hold_choice", map_hold), "hold", "go"), Optional(Choice("trapped_choice", map_door_trapped), "trapped", "not trapped"), ] defaults = { "color": "current", "hold": "go", "trapped": "not trapped", }

def _process_recognition(self, node, extras):
    color = extras["color"]
    hold = extras["hold"]
    trapped = extras["trapped"]
    print(f"{color} team {hold} wedge the {trapped} door")
    cmd_door_options(color, hold, "wedge", trapped).execute()

class RemoveTheWedge(CompoundRule): """ Speech recognise team remove the wedge """ spec = " [team] remove [the] (wedge | block) [from] [(the | that)] [door] [way]" extras = [ Optional(Choice("color_choice", map_colors), "color", "current"), Optional(Choice("hold_choice", map_hold), "hold", "go"), Optional(Choice("trapped_choice", map_door_trapped), "trapped", "not trapped"), ] defaults = { "color": "current", "hold": "go", "trapped": "not trapped", }

def _process_recognition(self, node, extras):
    color = extras["color"]
    hold = extras["hold"]
    trapped = extras["trapped"]
    print(f"{color} team {hold} remove the wedge from the {trapped} door")
    cmd_door_options(color, hold, "wedge", trapped).execute()

class UseTheWand(CompoundRule): """ Speech recognise team use the wand """ spec = " [team] use the (mirror | wand) [on] [(the | that)] [(door [way] | opening | room)]" extras = [ Optional(Choice("color_choice", map_colors), "color", "current"), Optional(Choice("hold_choice", map_hold), "hold", "go"), Optional(Choice("trapped_choice", map_door_trapped), "trapped", "not trapped"), ] defaults = { "color": "current", "hold": "go", "trapped": "not trapped", }

def _process_recognition(self, node, extras):
    color = extras["color"]
    hold = extras["hold"]
    trapped = extras["trapped"]
    print(f"{color} team {hold} use the wand on the {trapped} door")
    cmd_door_options(color, hold, "mirror", trapped).execute()

------------------------------------------------------------------

def cmd_stack_up(color, hold, side): """ Press & release command keys for team to stack up (on execution) """ actions = cmd_select_team(color) actions += map_ingame_key_bindings["cmd_menu"] actions += map_ingame_key_bindings["cmd_1"]

start hold for command

if hold == "hold":
    actions += action_hold("down")
# todo! in 1.0 some doors don't have all stack options available, update if/when Void update
match side: 
    case "split":
        actions += map_ingame_key_bindings["cmd_1"]
    case "left":
        actions += map_ingame_key_bindings["cmd_2"]
    case "right":
        actions += map_ingame_key_bindings["cmd_3"]
    case "auto":
        actions += map_ingame_key_bindings["cmd_4"]
# end hold for command
if hold == "hold":
    actions += action_hold("up")
return actions

class StackUp(CompoundRule): """ Speech recognise team stack up on door """ spec_start = " [team] " spec_1 = "stack " spec_2 = "stack [up] []" spec_3 = " stack" spec_end = "[(on (the | that) door [way] | there | here)]" spec = f"{spec_start} ({spec_1} | {spec_2} | {spec_3}) {spec_end}" extras = [ Optional(Choice("color_choice", map_colors), "color", "current"), Optional(Choice("hold_choice", map_hold), "hold", "go"), Choice("side", map_door_stack_sides), ] defaults = { "color": "current", "hold": "go",

todo! in 1.0 some doors don't have all stack options available, change to "auto" if/when Void update

    # keeping as "split" for now because it's cmd_1 and don't want to swap off primary weapon if stack options 
    # aren't available
    "side": "split", 
}

def _process_recognition(self, node, extras):
    color = extras["color"]
    hold = extras["hold"]
    side = extras["side"]
    print(f"{color} team {hold} stack up {side}")
    cmd_stack_up(color, hold, side).execute()

------------------------------------------------------------------

def cmd_breach_and_clear(color, hold, tool, grenade): """ Press & release command keys for team to breach & clear (on execution) """ actions = cmd_select_team(color) actions += map_ingame_key_bindings["cmd_menu"] if tool == "open": actions += map_ingame_key_bindings["cmd_2"] else: actions += map_ingame_key_bindings["cmd_3"] match tool: case "kick": actions += map_ingame_key_bindings["cmd_1"] case "shotgun": actions += map_ingame_key_bindings["cmd_2"] case "c2": actions += map_ingame_key_bindings["cmd_3"] case "ram": actions += map_ingame_key_bindings["cmd_4"] case "leader": actions += map_ingame_key_bindings["cmd_5"]

start hold for command

if hold == "hold":
    actions += action_hold("down")
match grenade:
    case "none":
        actions += map_ingame_key_bindings["cmd_1"]
    case "flashbang":
        actions += map_ingame_key_bindings["cmd_2"]
    case "stinger":
        actions += map_ingame_key_bindings["cmd_3"]
    case "gas":
        actions += map_ingame_key_bindings["cmd_4"]
    case "launcher":
        actions += map_ingame_key_bindings["cmd_5"]
    case "leader":
        actions += map_ingame_key_bindings["cmd_6"]
# end hold for command
if hold == "hold":
    actions += action_hold("up")
return actions

class BreachAndClear(CompoundRule): """ Speech recognise team breach and clear """

# "blue team on my command wait for my breach then clear it use flashbangs"
# "red on my order c2 the door use the fourty mil then breach and clear"
# "red team kick down the door breach and clear use cs"
# "gold open the door use flashbangs breach and clear"
# "blue flash and clear it"
# "red flash and clear"
# "kick it down clear it"
# "gold on my command c2 the door wait for my flash then breach and clear"
# "gold on my command shotgun the door lead will gas then clear it"
# "gold breach and clear use the fourty mil"
# "blue move in and clear it"

spec_start = "<color> [team] <hold>"
spec_tool = "[<tool>] [the door]"
spec_grenade = "[(throw | deploy | use)] <grenade> [grenade]"
spec_clear = "([then] breach and clear | (then | and) clear | [(then | and)] clear it | [then] move in and clear [it])"

spec = f"{spec_start} {spec_tool} ({spec_grenade} {spec_clear} | {spec_clear} {spec_grenade})"
extras = [
    Optional(Choice("color_choice", map_colors), "color", "current"),
    Optional(Choice("hold_choice", map_hold), "hold", "go"),
    Choice("tool", map_door_breach_tools),
    Optional(Choice("grenade_choice", map_door_grenades), "grenade", "none"),
]
defaults = {
    "color": "current",
    "hold": "go",
    "tool": "open",
    "grenade": "none",
}

def _process_recognition(self, node, extras):
    color = extras["color"]
    hold = extras["hold"]
    tool = extras["tool"]
    grenade = extras["grenade"]
    print(f"{color} team {hold} {tool} the door {grenade} breach and clear")
    cmd_breach_and_clear(color, hold, tool, grenade).execute()

------------------------------------------------------------------

def cmd_pick_lock(color, hold): """ Press & release command keys for the team to move to location

class PickLock(CompoundRule): """ Speech recognise team pick the lock """ spec = " [team] pick ([the] door | [the] lock | it)" extras = [ Optional(Choice("color_choice", map_colors), "color", "current"), Optional(Choice("hold_choice", map_hold), "hold", "go"), ] defaults = { "color": "current", "hold": "go", }

def _process_recognition(self, node, extras):
    color = extras["color"]
    hold = extras["hold"]
    print(f"{color} team {hold} pick the lock")
    cmd_pick_lock(color, hold).execute()

------------------------------------------------------------------

def cmd_ground_options(color, hold, ground_option): """ Press & release command keys for the team to move, cover, halt (hold), search area

class GroundOptions(CompoundRule): """ Speech recognise team move, cover, halt (hold), search area """ spec = " [team] " extras = [ Optional(Choice("color_choice", map_colors), "color", "current"), Optional(Choice("hold_choice", map_hold), "hold", "go"), Choice("ground_option", map_ground_options), ] defaults = { "color": "current", "hold": "go", "ground_option": "move" }

def _process_recognition(self, node, extras):
    color = extras["color"]
    hold = extras["hold"]
    ground_option = extras["ground_option"]
    print(f"{color} team {hold} {ground_option}")
    cmd_ground_options(color, hold, ground_option).execute()

------------------------------------------------------------------

def cmd_fallin(color, hold, formation): """ Press & release command keys for team to fall in (on execution)

class FallIn(CompoundRule): """ Speech recognise team fall in """ spec_1 = " [team] (fall in | regroup | form up) [on me] []" spec_2 = " [team] on me []" spec = f"({spec_1} | {spec_2})" extras = [ Optional(Choice("color_choice", map_colors), "color", "current"), Optional(Choice("hold_choice", map_hold), "hold", "go"), Choice("formation", map_ground_fallin_formations), ] defaults = { "color": "current", "hold": "go", "formation": "single", }

def _process_recognition(self, node, extras):
    color = extras["color"]
    hold = extras["hold"]
    formation = extras["formation"]
    print(f"{color} team {hold} fall in {formation}")
    cmd_fallin(color, hold, formation).execute()

------------------------------------------------------------------

def cmd_use_deployable(color, hold, deployable): """ Press & release command keys for the team to use deployable (on execution)

class UseDeployable(CompoundRule): """ Speech recognise command team to use a deployable at a location """ spec = " [team] deploy " extras = [ Optional(Choice("color_choice", map_colors), "color", "current"), Optional(Choice("hold_choice", map_hold), "hold", "go"), Choice("deployable", map_ground_deployables), ] defaults = { "color": "current", "hold": "go", "deployable": "flashbang", }

def _process_recognition(self, node, extras):
    color = extras["color"]
    hold = extras["hold"]
    deployable = extras["deployable"]
    print(f"{color} team {hold} deploy {deployable}")
    cmd_use_deployable(color, hold, deployable).execute()

------------------------------------------------------------------

def cmd_npc_player_interact(interaction): """ Press & release command keys for player to interact with target (on execution)

class NpcPlayerInteract(CompoundRule): """ Speech recognise command an NPC (not team) """ spec = "you " extras = [ Choice("interaction", map_npc_player_interacts), ]

def _process_recognition(self, node, extras):
    interaction = extras["interaction"]
    print(f"player to NPC {interaction}")
    cmd_npc_player_interact(interaction).execute()

------------------------------------------------------------------

def cmd_npc_team_restrain(color): """ Press & release command keys for team to restrain NPC target (on execution)

class NpcTeamRestrain(CompoundRule): """ Speech recognise command team to restrain NPC target """ spec_start = " [team]" spec_1 = " (em | them | him | her | [the] target)" spec = f"{spec_start} {spec_1}" extras = [ Optional(Choice("color_choice", map_colors), "color", "current"), Choice("restrain", map_npc_team_restrain), ] defaults = { "color": "current", "restrain": "restrain", }

def _process_recognition(self, node, extras):
    color = extras["color"]
    print(f"{color} team restrain target")
    cmd_npc_team_restrain(color).execute()

------------------------------------------------------------------

def cmd_npc_team_deploy(color, deployable): """ Press & release command keys for team to use deployable on NPC target (on execution)

class NpcTeamDeploy(CompoundRule): """ Speech recognise command team to use deployable on NPC target """ spec_start = " [team]" spec_target = "(em | them | him | her | [the] target)" spec_1 = f"subdue {spec_target} [(use | with)] []" spec_2 = f" {spec_target}" spec_3 = f"make {spec_target} compliant [(use | with)] []" spec = f"{spec_start} ({spec_1} | {spec_2} | {spec_3})" extras = [ Optional(Choice("color_choice", map_colors), "color", "current"), Choice("deployable", map_npc_team_deployables), ] defaults = { "color": "current", "deployable": "melee", }

def _process_recognition(self, node, extras):
    color = extras["color"]
    deployable = extras["deployable"]
    print(f"{color} team {deployable} target")
    cmd_npc_team_deploy(color, deployable).execute()

------------------------------------------------------------------

def cmd_select_team_member(team_member): """ Press & release select team member key (on execution) """ return map_ingame_key_bindings[team_member]

class SelectTeamMember(CompoundRule): """ Speech recognise commands to individual team member """ spec = "" extras=[ Choice("team_member", map_team_members), ]

def _process_recognition(self, node, extras):
    team_member = extras["team_member"]
    print(f"Select {team_member}")
    cmd_select_team_member(team_member).execute()

------------------------------------------------------------------

def cmd_team_member_options(team_member, option, additional_option): """ Press & release command keys for interacting with individual team member (on execution) """ actions = map_ingame_key_bindings[team_member] actions += map_ingame_key_bindings["cmd_menu"] match option: case "move": actions += map_ingame_key_bindings["cmd_1"] match additional_option: case "here": actions += map_ingame_key_bindings["cmd_1"] case "here then back": actions += map_ingame_key_bindings["cmd_2"] case "focus": actions += map_ingame_key_bindings["cmd_2"] match additional_option: case "here": actions += map_ingame_key_bindings["cmd_1"] case "my position": actions += map_ingame_key_bindings["cmd_2"] case "door": actions += map_ingame_key_bindings["cmd_3"] case "target": actions += map_ingame_key_bindings["cmd_4"] case "unfocus": actions += map_ingame_key_bindings["cmd_5"] case "unfocus": actions += map_ingame_key_bindings["cmd_2"] actions += map_ingame_key_bindings["cmd_5"] case "swap": actions += map_ingame_key_bindings["cmd_3"] match additional_option: case "alpha": actions += map_ingame_key_bindings["cmd_1"] case "bravo": actions += map_ingame_key_bindings["cmd_2"] case "charlie": actions += map_ingame_key_bindings["cmd_3"] case "delta": actions += map_ingame_key_bindings["cmd_4"] case "search": actions += map_ingame_key_bindings["cmd_4"] return actions

class TeamMemberOptions(CompoundRule): """ Speech recognise commands to individual team member """ spec = "

def _process_recognition(self, node, extras):
    team_member = extras["team_member"]
    option = extras["option"]
    move_option = extras.get("move_option")
    focus_option = extras.get("focus_option")
    other_team_member = extras.get("other_team_member")
    additional_option = ((move_option if move_option is not None else "") 
        + (focus_option if focus_option is not None else "")
        + (other_team_member if other_team_member is not None else ""))
    print(f"{team_member} {option} {additional_option}")
    cmd_team_member_options(team_member, option, additional_option).execute()

------------------------------------------------------------------

def cmd_yell(): """ Press & release yell key (on execution) """ return map_ingame_key_bindings["yell"]

class YellFreeze(BasicRule): """ Speech recognise yell at NPC """ element = Alternative(( Literal("freeze"), Literal("hands"), Literal("drop"), Literal("drop it"), Literal("police"), ))

def _process_recognition(self, node, extras):
    print("Freeze!")
    cmd_yell().execute()

------------------------------------------------------------------

class NoiseSink(MappingRule): """ Capture any other noises or words outside of commands, and do nothing """ mapping = {'': ActionBase()} extras = [ Dictation("dictation") ]

---------------------------------------------------------------------------

Recognition Observer - for mid-utterance recognition

class FreezeRecob(RecognitionObserver): """ Observer of partial recognition of yell commands """ def init(self): RecognitionObserver.init(self) self.words = None self.frozen = False

def on_begin(self):
    self.frozen = False

def on_partial_recognition(self, words, rule):
    self.words = words
    if (not self.frozen) and isinstance(rule, KaldiRule) and rule.name == "ReadyOrNot_priority::YellFreeze":
        print("Freeze!")
        cmd_yell().execute()
        self.frozen = True

def on_recognition(self, words, results, rule, node):
    self.words = words
    # print("words={0}".format(words))
    # print("results={0}".format(results))

def on_failure(self, results):
    self.words = False

def on_end(self, results):
    self.words = False
    self.frozen = False

---------------------------------------------------------------------------

Add rules to grammar and create RecognitionObserver instances

grammar.add_rule(ExecuteOrCancelHeldOrder()) grammar.add_rule(SelectTeam()) grammar.add_rule(SelectColor()) grammar.add_rule(DoorOptions()) grammar.add_rule(WedgeIt()) grammar.add_rule(RemoveTheWedge()) grammar.add_rule(UseTheWand()) grammar.add_rule(StackUp()) grammar.add_rule(BreachAndClear()) grammar.add_rule(PickLock()) grammar.add_rule(GroundOptions()) grammar.add_rule(FallIn()) grammar.add_rule(UseDeployable()) grammar.add_rule(NpcPlayerInteract()) grammar.add_rule(NpcTeamRestrain()) grammar.add_rule(NpcTeamDeploy())

grammar.add_rule(TeamMemberOptions()) # needs key bindings for alpha-delta in-game

grammar.add_rule(SelectTeamMember()) # needs key bindings for alpha-delta in-game

grammar_priority.add_rule(YellFreeze()) if USE_NOISE_SINK: grammar_priority.add_rule(NoiseSink())

freeze_recob = FreezeRecob()

---------------------------------------------------------------------------

Load the grammar instance, register RecognitionObservers, and define how

to unload them.

grammar.load() grammar_priority.load() freeze_recob.register()

---------------------------------------------------------------------------

if DEBUG_MODE: from lark import Lark, Token import itertools grammar_string = r""" ?start: alternative

// ? means that the rule will be inlined iff there is a single child ?alternative: sequence ("|" sequence) ?sequence: single | sequence "{" WORD "}" -> special

?single: WORD+ -> literal | "<" WORD ">" -> reference | "[" alternative "]" -> optional | "(" alternative ")"

// Match anything which is not whitespace or a control character, // we will let the engine handle invalid words WORD: /[^\s[]<>|(){}]+/

%import common.WS_INLINE %ignore WS_INLINE """

def do_on_tree_item(tree_item):
    elements = []
    if tree_item.data == "literal":
        literal_children = []
        for child in tree_item.children:
            literal_children.append(child)
        elements.append(' '.join(literal_children))
        literal_children = None
        return elements
    if tree_item.data == "optional": 
        elements.append("")
        for child in tree_item.children:
            if child is None: 
                continue
            elements.extend(do_on_tree_item(child))
        return elements
    if tree_item.data == "alternative": 
        for child in tree_item.children:
            if child is None: 
                continue
            elements.extend(do_on_tree_item(child))
        return elements
    if tree_item.data == "sequence": 
        for child in tree_item.children:
            if child is None: 
                continue
            elements.append(do_on_tree_item(child))
        product_iter = itertools.product(*elements)
        product_list = [' '.join((' '.join(i)).split()) for i in product_iter]
        product_set = set(product_list)
        product_list = list(product_set)
        product_set = None
        return product_list

with open(".debug_grammar_readyornot.txt", "w") as file:
    file.write(grammar.get_complexity_string())
    file.write(f"\n{grammar_priority.get_complexity_string()}\n")

    for rule in grammar.rules:
        file.write(f"\n\n---{rule.name}---")
        file.write(f"\n{rule.element.gstring()}")
        file.write(f"\n---")

        # file.write(f"\n{rule._element.element_tree_string()}")
        # file.write(f"\n---")

        spec_parser = Lark(grammar_string, parser="lalr")
        tree = spec_parser.parse(rule.element.gstring())
        # file.write(f"\n{tree.pretty()}")

        if DEBUG_HEAVY_DUMP_GRAMMAR: 
            # do_on_tree_item() can be expensive on memory, so we don't do this for 
            # just DEBUG_MODE
            for tree_item in tree.children:
                tree_item_options = do_on_tree_item(tree_item)
                file.write(f"\n{tree_item_options}")
            file.write(f"\n---")
        try:
            if hasattr(rule, 'spec'):
                file.write(f"\n{rule.spec}")
            if hasattr(rule, 'extras'):
                for extra in rule.extras:
                    if isinstance(extra, Choice):
                        choice_name = extra.name
                        choice_keys = list(extra._choices.keys())
                        file.write(f"\n{choice_name}={choice_keys}")
                    elif isinstance(extra, Optional):
                        if isinstance(extra._child, Choice):
                            el_name = extra.name
                            choice_keys = list(extra._child._choices.keys())
                            file.write(f"\n{el_name}=Optional({choice_keys})")
                    else:
                        el_name = extra.name
                        el_tree = extra.element_tree_string()
                        file.write(f"\n{el_name}={el_tree}")
        except Exception: 
            # it doesn't matter if we can't dump the grammar into a file & it may fail
            # if rules are added that don't only use CompoundRule and Choice
            print(f"Unable to grammar dump all of {rule.name}")
            pass
        file.write(f"\n------------")

    for rule in grammar_priority.rules:
        file.write(f"\n\n{rule.element.gstring()}")

Unload function which will be called at unload time.

def unload(): global grammar global grammar_priority global freeze_recob if grammar: grammar.unload() grammar = None if grammar_priority: grammar_priority.unload() grammar_priority = None freeze_recob.unregister() freeze_recob = None

ShadowESH123 commented 3 months ago

Is there need to specify the game folder? in the settings is there need for "," after the debug mode line?

ShadowESH123 commented 3 months ago

I got it to work! Its amazing thanks for the work