search

talonvoice / talon

Issue Tracker for the main Talon app
85 stars 0 forks source link

Thoughts on Subtitles #348

Open jptreen opened 3 years ago

jptreen commented 3 years ago

The subtitles implemented in beta give two valuable pieces of feedback instantly:

1) A command has been received! 2) Is it the right one?

I would like to be able to control the way these subtitles display. I love them, do not want to disable them, but if I find myself working at the bottom of the screen for some reason saying small phrases or commands (and I often do), then the couple of words I dictate frequently get in the way of my work.

Talonfile Actions

It would make it a more useful feature if actions were mapped so we can set while in the app (for example):

This should allow the subtitles to be useful no matter which application you are using.

Very pleasing example of when it works...

image

Somewhat less pleasing example...

image

A couple of examples of why this might be useful:

Subtitles are (almost?) completely pointless in dication mode, for instance. What you type is what you get.

Now, on the other hand, if you are trying to run Photoshop, an application with an insane amount of commands, many of which won't give you any visual feedback, it would be really useful to know, well... whether the command has been received, and is it the one you were trying to make.

It might be that the right command had been received, but the command surprised you by not actually doing anything. Without feedback, you might just keep trying before getting frustrated. With the subtitle, yes you get frustrated, but at least you're frustrated by your photoshop fluency, rather than Talon.

pokey commented 3 years ago

I would also kindly request the ability to control for how long the subtitles are displayed

adam-edison commented 2 years ago

We should also be able to pick the background - transparent or opaque (maybe an alpha value?) and of course the color. For some apps, without a background the text will not be visible very easily (e.g. a mostly white screen).

lunixbochs commented 2 years ago

The text is visible on a white background

adam-edison commented 2 years ago

@lunixbochs

image

Notice for example in Amazon Prime Video, how the default option is much easier to read than Preset 1:

image

jgarvin commented 2 years ago

I don't want to give a laundry list of nice-to-haves because I know developer time is finite, so I'll just give the three things that I think would be the most immediately useful and help users begin to start exploring the space of what's possible:

I don't think there's any getting around the fact that what font size is appropriate is subjective and is going to be based on the user's particular display and eye sight. I just don't I think it's reasonable or possible for talon to be able to pick a size that's perfect for everybody. I realize the font already shrinks with the length of the utterance in order to fit on screen, but perhaps users can specify a starting size, and then the font can be shrunk from there if need be for particularly long utterances.

I think this might be an area where giving people control in scripts might lead to interesting ideas from the community. I know I would be interested in making it so that the subtitles appear within the window that currently has focus, because that's where my eyes usually already are. I've also seen other people in slack working on projects to scan the screen to identify interface elements, which they could maybe use to influence subtitle position.

Users with vision impairment may have trouble perceiving color differences, separate from whether or not the font is big enough. I'm not any kind of eye doctor so I'm not sure why, but I know the high contrast gui color themes usually use yellow instead of white, and if you buy high contrast glasses in order to keep your eyes more relaxed they are also usually yellow tinted.

adam-edison commented 2 years ago

@jgarvin Yellow tinted glasses are to block blue light - not related. Yellow subtitles are often against black backgrounds for high contrast but lower brightness.

rntz commented 2 years ago

I'd like to specifically bump the request for actions to turn subtitles on/off. (It would be similarly convenient to have actions to turn recording on/off.)

adam-edison commented 2 years ago

@rntz It appears that in the beta, both the option to turn subtitles on or off, and the option to turn recording on or off, are already part of the menu:

image

I'm not sure when this will make it to non-beta.

rntz commented 2 years ago

I know, I'm in the beta; I'm asking for Talonscript-accessible actions, so that one could define a voice command to enable/disable/toggle subtitles/recordings.

On Sun, Sep 12, 2021 at 17:42 Adam Edison @.***> wrote:

@rntz https://github.com/rntz It appears that in the beta, both the option to turn subtitles on or off, and the option to turn recording on or off, are already part of the menu:

[image: image] https://user-images.githubusercontent.com/15655308/132995739-570ab116-2031-4794-ab29-3d8f4068afcb.png

I'm not sure when this will make it to non-beta.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/talonvoice/talon/issues/348#issuecomment-917669444, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAD6DYDXAWZCITRIDZ6BG3UBTJ7JANCNFSM5AEU4A3A .

adam-edison commented 2 years ago

@rntz In settings.talon, we can toggle recordings on/off.

    # record speech as audio files for recognized input
    # speech.record_all = 1

I don't think anything is exposed for subtitles yet

lunixbochs commented 2 years ago

subtitles also have a setting, speech._subtitles, with the underscore indicating it's subject to change. You can toggle these both by using setting the settings when in specific modes, or using other contextual information.

wolfmanstout commented 2 years ago

+1 to control over subtitle font size. Folks on this thread may also be interested in talon_hud: https://github.com/chaosparrot/talon_hud

I've started to disable subtitles and use the "event log" in talon_hud and I'm quite happy with it. It has other nice features too like a mode indicator (sleep/command/dictation) and automatically turns off the event log in dictation mode.

AndreasArvidsson commented 2 years ago

Support for subtitles on other screens than main is also requested.

pokey commented 2 years ago

We'd also like to add the ability to change the text that shows for subtitles. We would have a subtitles-specific action like "format_text" that we could use to influence the text that's displayed

pokey commented 2 years ago

Fwiw it would be nice if we could use the same action to customize the command history; would basically be moving knausj85/knausj_talon#758's history_transform_phrase_text to be a core action

richardmcmillen commented 2 years ago

This is also something I am very interested in. Subtitle positioning is the top priority for me however, for the same reason as it has already been outlined that the subtitle position conflicts with Slack and other applications where I am typically interacting with the bottom portion.

Flexibility would be great but I think even just being able to have a fixed position for all applications would be okay, although ideally this could be context specific.

ziemus commented 2 years ago

It is a minor inconvenience but I think that turning off or removing the present subtitles when capturing a screenshot is a good idea - now all screenshot appear with the 'grab xyz' commands. And if there were a toggle that could turn this feature on and off in the HUD, everyone could customize the behavior to their own preference. Making it viable to programmatically set the subtitles on and off and clear them on runtime could also solve this issue.