justin-roche / serenade-mode

Emacs minor mode providing integration with Serenade
18 stars 3 forks source link

Serenade Mode

Serenade mode is a minor mode allowing voice-based structure editing and control of Emacs through integration of Serenade. Serenade-mode features:

Why another Emacs voice control mode?

Emacs has a long and distinguished history of voice control modes, summarized at EmacsWiki. However, many are no longer accessible, and those that are suffer from one or more the following drawbacks:

In contrast to these, Serenade mode aims to be a light wrapper allowing interaction with a third-party voice-recognition and structure editing tool, so that the heavy lifting is done outside of Emacs, and Emacs can focus on doing what it does best: providing a fully customizable user interface for text-based workflows.

The guiding principles for the design of the mode are:

Installation

First, download Serenade and start the application.

Next, download the serenade-mode source code here, add the serenade-mode directory to your Emacs load path, and call:

(use-package serenade-mode)

Getting Started

To enable helm and yasnippet, set the following variables:

(setq serenade-completion-frontend 'helm)
(setq serenade-helm-M-x t)
(setq serenade-snippet-engine 'yasnippet)

To start the the mode call:

(serenade-mode)

If all goes well, you should see the active application change to Emacs in the Serenade application's overlay window.

Command Taxonomy

The implementation of a generated command includes the following components:

Multiple speech patterns can be associated with a command (synonyms), but a single speech pattern can be associated with only one command per speech map.

Built-In default speech bindings

Built-In bindings are those that specify bindings for Serenade's built-in commands. Serenade already recognizes these commands and sends specific messages to the plugin. The map for these is found in serenade-defaults.el. All bindings besides these involve the configuration of javascript in the Serenade directory.

Pattern Command
close tab delete-window
undo serenade--undo
copy serenade--copy-selection
cut serenade--cut-selection
select serenade--select-target
undo serenade--undo
redo serenade--redo
open serenade--open-file
tab serenade--switch-tab
close tab delete-window
save save-buffer
create tab split-window-right-and-focus
next tab next-buffer
previous tab previous-buffer
scroll scroll-up-command
scroll down scroll-up-command
scroll up scroll-down-command
open file list list-buffers
style nil

Generated default speech bindings

The snippet \<name> command inserts a Yasnippet snippet of that name. snippet \<name> of \<arg> autofills the first field of the snippet with \<arg>.

Pattern Command
show commands serenade-commands
serenade log serenade-commands-log-open-log
snippet \<name> of \<arg> serenade--insert-yasnippet-with-args
snippet \<name> serenade--insert-yasnippet

Customization

Speech Bindings

To add a speech binding to a speech map, call serenade-define-speech with the symbol for the map, the speech pattern, and the associated command. If a map does not exist, it will be created. Serenade mode must be restarted or the function serenade-generate must be called for these customizations to take effect. It is possible to bind any speech pattern that does not conflict with Serenade commands.


   (serenade-define-speech 'global "treemacs" 'treemacs) 

   (serenade-define-speech 'org-mode "promote" 'org-do-promote) 

To add variables to the speech pattern, enclose them in arrow brackets. Arguments are passed in the order specified in the speech pattern.


   (serenade-define-speech 'global "open buffer <name>" 'switch-to-buffer) 

To reorder the input of arguments to the command, include %n as an argument transformer in the pattern definition:


   (serenade-define-speech 'global "open buffer <%2 name> <%1 direction>" 'another-switch-buffer-fn) 

It is possible to use an alist as the second argument to define-speech:

(serenade-define-speech 'org-mode '(("promote" . org-do-promote) 
                                    ("demote" . org-do-demote)))

The convenience function serenade-global-set-speech has the same signature as define-speech, but applies to the global map only:


(serenade-global-set-speech '( ("a" . b) ))

Macros

To help ensure discoverability, speech maps do not allow lambdas as bound commands. You can instead use the provided currying macro serc within a backquoted list, shown here with its macro expansion:

(serenade-define-speech 'org-mode `(("switch to special buffer" . ,(serc switch-to-buffer "special"))))

=>

(serenade-define-speech 'org-mode `(("switch to special buffer" . 'serenade-curried->switch-to-buffer->special)))

The currying macro is compatabile with speech pattern variables, which are applied as the final arguments to the curried function.

There is also the provided serd macro, which acts like a defun call but returns the symbol of the new function, allowing you to inline function definitions in the speech map.


(serenade-define-speech 'global `(("a <n>" . ,(serd custom-fn(a) 
                                                    (setq test-val a) )))) 

=>

(serenade-define-speech 'global `(("a <n>" . ,(defun custom-fn
                                                  (a)
                                                (setq test-val a)
                                                (intern-soft 'custom-fn))))) 

To inline interactive function calls, use the seri macro, which simply returns a symbol for the interactive invocation of the function:


("switch workspace" . ,(seri treemacs-switch-workspace)) 

=>

("switch workspace" . ,(intern-soft 'serenade-interactive->treemacs-switch-workspace)) 

Mode configuration

For some modes it can be useful to specify configurations that are not specific to particular speech bindings. For this there is the function serenade-configure-mode. This can be used to expose only a portion of the buffer as editable (as with shell-mode), or for cleanup:


(serenade--configure-mode :mode 'rjsx-mode 
                          :post-edit 'js2-reparse)                                    

For more about the mode configuration options, see the documentation in the source code.

Variables

serenade-mode-filetypes

This variable specifies filetypes that can be used as serenade buffers, which are buffers subject to Serenade's editing operations. By default this includes all the filetypes Serenade handles, with the addition of Elisp files.

  '("js" "py" "c" "h" "cpp" "cc" "cxx" "c++" "hpp" "hh" "hxx" "h++""cs""css" "scss""dart" "go" "html" "vue" "svelte" "java" "js" "jsx" "jsx" "js""jsx" "js" "kt" "py" "rb" "rs" "scss" "sh" "bash" "ts" "tsx" "tsx" "ts""vue" "html" "el")

serenade-directory

To manage custom commands, serenade-mode autogenerates javascript stored in the serenade scripts directory. The location of this directory is specfied by serenade-directory with the default value:

"~/.serenade/scripts/"

serenade-sync-on-start

This variable specifies whether the autogeneration of custom javascript should happen each time the mode is started. Default is true.

serenade-auto-set-evil

This specifies whether certain buffer editing commands integrate with evil or default Emacs editing commands, based on the presence of evil mode. Default is true.

serenade-helm-completion-frontend

Sets the completion frontend to be used. If nil, completion support is disabled. If 'helm, uses helm for completion.

serenade-helm-M-x

If true, serenade-mode advices helm-M-x so that speech patterns appear beside the keybinding for M-x commands. Default value is nil.

serenade-snippet-engine

Sets the snippet engine to be used. Currently the only supported value is 'yasnippet. If nil, snippet support is disabled.

serenade--add-builtin-global-defaults

this variable determines whether the global defaults are added when the mode loads.

serenade--add-generated-global-defaults

this variable determines whether the generated global defaults are added when the mode loads.

Hooks

serenade-mode-init-hook

The list of functions to be called after `serenade-mode' has initialized all variables, before connecting fer for the first time.

serenade-mode-after-edit-hook

The list of functions to be called after an edit has been made in response to a speech command.

serenade-double-line-numbers-on/serenade-double-line-numbers-on

For voice coding it can be useful to display both relative and absolute line numbers simultaneously. Associated hooks are provided to allow customization of this operation. These hooks run if serenade--enable-double-line-numbers is true.

Functions

(serenade-generate)

Convert custom commands added to serenade speech bindings to javascript.

(serenade-helm-commands)

This function displays all the currently bound serenade-mode commands in a helm buffer.

(serenade-helm-active-commands)

This function displays all the currently bound and active serenade-mode commands in a helm buffer, ie. commands which, if given, map to an active speech map.

(serenade-helm-selectors)

This function displays displays a reference list of serenade selectors, such as "parameter", "argument", "method", etc. in a helm buffer.

(serenade--log-open-log)

This function displays the log for serenade-mode.

Limitations

Under the current implementation of generated commands, the mode bindings do not constrain what the speech engine is listening for. The speech engine listens for all commands, and only once the command reaches Emacs is the correct command selected. This may or may not have performance effects on the recognition accuracy of the engine if there are many similar sounding commands.

Contributing

Pull requests, issues, and feature requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

License

GNU General Public License