Command Wizard Show works, I can choose either 1, 2, or 3. But then nothing happens.

[therealrobster]

Hi there,

I am trying to give this tool a try. It looks like the exact solution to a problem I am having whilst navigating with Talon.

I have installed it. Talon loads it, and gives no errors.

I used to the "Command Wizard Show" command. It brings up the help menu where I can choose one two or three. However, when I choose one (or the other two options), nothing happens.

The Command Wizard Help Menu simply disappears after I choose an option, and that is that.

I am not seeing any errors in the terminal, so for now I cannot get it to work unfortunately.

I am running X11 on Linux (popOS!), using Gnome, if that helps, or makes any difference?

Is there anything I can try, or test, to get this working?

Thank you so much for your time, I do appreciate it.

[splondike]

I suspect you may have some kind of canvas issue, but let's test that. Make a new file called e.g. test.py in your Talon user directory and put this in it:

from talon import actions
from talon.types import Rect

actions.user.marker_ui_show([
    Rect(0, 0, 100, 100)
])

This exercises some code in this repo. If it works you'll see a small letter on a cyan background near the top left of one of your screens. If you don't see it, try this code instead:

from talon import canvas
from talon.types import Rect

def _draw(can):
    can.paint.color = "red"
    can.draw_rect(rect)

rect = Rect(0, 0, 100, 100)
can = canvas.Canvas.from_rect(rect)
can.register("draw", _draw)
can.show()
can.freeze()

This just uses the basic Talon canvas features, in this case we'd want to see a red rectangle. If that doesn't work you might need to ask for some help on Slack.

If the first cyan letter test works then remove the existing tests and put the following file in the same folder as talon_ui_helper (e.g. in issuetest.py next to the overlays.py file). It should directly try to show one of the overlays. Hopefully then we'll get a success, or something in the console:

from .overlays import ImageSelectorOverlay

ImageSelectorOverlay(lambda x: None)

[therealrobster]

Thank you so much for your highly detailed response, I do appreciate it.

I ran the first test and it did work, a small letter a in a cyan square appeared near the top left of my screen.

I then created the file as suggested at the end of your post and named it issuetest.py, and when I tried to run the command wizard show command, then choose an option, I have the following output now in the terminal:

2024-10-17 09:28:50.442 DEBUG [+] /home/username/talon/user/talon_ui_helper/issuetest.py
2024-10-17 09:29:26.944 WARNING DFA rule missing: <user.number>

The first line is when I created the file and pressed save, and Talon showed that it loaded. The second line shows whenever I say "Choose One", from the Command Wizard Show menu.

Does the error mean anything to you?

Again, thank you so much for your time.

[therealrobster]

Just an update on this, I completely exited Talon, and restarted it to see if it would do anything.

Now, it does not show the warning about the DFA missing at all.

[therealrobster]

And another quick update, and I'm not sure if this helps.

I just ran a whole heap of repetitive "Command Wizard Show", then worked my way through options 1 to 3. I did this about twenty times.

The reason I did this, is that sometimes when I choose one of the options, and it doesn't have any pattern as to which option, the screen flickers very quickly, and I can see something appear and then disappear. It is so fast that I can't make out what it is, and nothing is left in the terminal.

I suspect that it is trying to show the interface, and then for some reason it closes before it gets a chance for me to even see what it is. I hope that helps in some way?

[splondike]

That flickering does sound like a helpful clue. If we go with the hypothesis that the canvas is starting up and then quickly being closed I can see a few options in the code: the focus event handler closes it when it gets an unfocus event, and the keypress handler closes it when enter or escape are pressed.

I suspect it's the focus handler. I think I put this in because if the canvas isn't focused at the start you can't give it focus by clicking on it or alt-tab-ing or anything like that. So you end up with a full screen thing blocking your use of the computer.

In any case we can see if it's the issue. Can you edit the overlays.py file on your version and put a print("Focus event", focussed) line just before line 159. It should be the same indent level as the if not focussed one that comes after it (use spaces to make them the same level).

After doing that, try rerunning the issuetest.py file. We should hopefully get some Focus event messages in the log that will tell us what the code is dealing with. If we don't you may like to add a print statement like print("Key event", evt) before line 168 (again making sure the indentation matches the following line). Then rerun the test, and we might see some key press events written to the log.

As an aside, how familiar are you with software programming? I can tune how explicit I make my instructions based on that.

[therealrobster]

Thank you so much for again such detailed feedback, and suggestions.

Regarding my programming level, I would consider myself a beginner. I use python regularly to write data conversion tools, and I dabble a little bit in pandas as part of this. But this is hobbyist level stuff. I can wrangle docker containers, have my conversion tools in Flask via docker, and technically quite OK with the computer in general. I understand functions, and making function calls, returning data, and all of that kind of stuff.

OK, I took the line print("Focus event", focussed) and added it to the appropriate place.

As soon as I pressed save on the overlays.py in my text editor I got the following output, in the terminal:

2024-10-18 10:31:20.160 DEBUG [~] user.talon_ui_helper.overlays: reloading import chain
2024-10-18 10:31:20.161 DEBUG [~] (user.talon_ui_helper.overlays) -> user.talon_ui_helper.issuetest
2024-10-18 10:31:20.393 DEBUG [~] (user.talon_ui_helper.overlays) -> user.talon_ui_helper.command_wizard
2024-10-18 10:31:20.734    IO Focus event False

Just to test, I pressed save again, and got the following:

2024-10-18 10:34:00.165 DEBUG [~] /home/userName/.talon/user/talon_ui_helper/overlays.py
2024-10-18 10:34:00.171 DEBUG [~] user.talon_ui_helper.overlays: reloading import chain
2024-10-18 10:34:00.171 DEBUG [~] (user.talon_ui_helper.overlays) -> user.talon_ui_helper.issuetest
2024-10-18 10:34:00.438 DEBUG [~] (user.talon_ui_helper.overlays) -> user.talon_ui_helper.command_wizard
2024-10-18 10:34:00.792    IO Focus event False

I then ran the command Command Wizard Show, and worked my way through the three options. Each time, nothing happened as before, but the terminal outputted:

2024-10-18 10:36:01.710    IO Focus event False
2024-10-18 10:36:07.420    IO Focus event False
2024-10-18 10:36:12.265    IO Focus event False

It was one line per button.

Just to be thorough, I took your line of code, print("Key event", evt), and added it to the appropriate place as you suggested. Again, as soon as I pressed save:

2024-10-18 10:40:44.148 DEBUG [~] /home/userName/.talon/user/talon_ui_helper/overlays.py
2024-10-18 10:40:44.154 DEBUG [~] user.talon_ui_helper.overlays: reloading import chain
2024-10-18 10:40:44.154 DEBUG [~] (user.talon_ui_helper.overlays) -> user.talon_ui_helper.issuetest
2024-10-18 10:40:44.402 DEBUG [~] (user.talon_ui_helper.overlays) -> user.talon_ui_helper.command_wizard
2024-10-18 10:40:44.789    IO Focus event False

Then again, as I went through the command wizard show, and went through three buttons, I got the following:

2024-10-18 10:39:42.093    IO Focus event False
2024-10-18 10:39:51.005    IO Focus event False
2024-10-18 10:39:56.332    IO Focus event False

I have no idea if that helps in any way?

[splondike]

Yeah, so sounds like that focus=False event is what's closing it. You can turn off that feature by commenting out the if not focussed block in the _focus_event handler.

If you leave in the print statement then we can see if it gets any other focus=False events. It might just be that I need to put in a delay before it starts closing the window on focus=False events.

[therealrobster]

Thank you, I just commented that out and as soon as I pressed save I had this popup.

If I move my mouse towards it it switches to the top of the screen if I move my mouse to the top of the screen it switches to the bottom of the screen.

The only way to get rid of it was to uncomment those lines and save again.

I am sorry for the trouble, and I do appreciate your help here.

[splondike]

So that popup probably comes from our issuetest.py test case earlier. A few follow on questions:

• Pressing escape didn't work to close it? If not, that would imply the canvas doesn't have focus (and might explain the focus=False events).
• Did you see if the overlay responded to click and drag events?
• Did the rest of the screen become darker and stop updating what was shown in applications? I.e. did the overlay appear to visually work?
• Did you get any messages from our print statements in the terminal?

[therealrobster]

Thank you again for your continued investigation.

• Pressing escape did not work to close it.
• The overlay did respond to click and drag, and created a square box selector when I dragged. I should have mentioned this I do apologize.
• The rest of the screen does become darker and stop updating. All I could do was move the mouse and create a square box selector when I dragged.
• RE messages: below is the output from the moment I pressed save in my text editor.

2024-10-23 09:09:26.340 DEBUG [~] /home/username/.talon/user/talon_ui_helper/overlays.py
2024-10-23 09:09:26.347 DEBUG [~] user.talon_ui_helper.overlays: reloading import chain
2024-10-23 09:09:26.347 DEBUG [~] (user.talon_ui_helper.overlays) -> user.talon_ui_helper.issuetest
2024-10-23 09:09:26.566 DEBUG [~] (user.talon_ui_helper.overlays) -> user.talon_ui_helper.command_wizard
2024-10-23 09:09:26.912    IO Focus event False

I hope that helps.

[splondike]

I went ahead and installed PopOS in a virtual machine and tested with the latest Talon beta (0.4.0-444-gc682) and public (0.4.0) against the PopOS default WM (GNOME3) and Xfce4. I could reproduce the above issue with both. I also gave it a go on OSX, and that worked fine. I suspect at some point the behaviour of Talon on Linux changed and this package stopped working there; I use Linux myself so it certainly worked last time I used it (admittedly a couple of years ago)!

In terms of what to do about it I had a bit of a play around and wasn't able to find a quick fix for the issue. I think the options are as follows:

• Since the mouse interraction still works I could change the control of the overlay to not use keyboard anymore; this would mean adding voice commands to this Talon user file set and probably some clickable widgets for non-voice users. Clickable widgets may get in the way of choosing a part of the screen to select, so there would need to be a way to move them somewhere else if they got in the way.
• Set the panel = True property on the canvas. I was avoiding this previously because it means the canvas can no longer cover the WM toolbar (e.g. the clock, logout widgets in GNOME3). That means that if you wanted to select items from those we'd either not be able to do that, or would have to do something like scale the screenshot of the full screen to fit the smaller available window size, which would distort it.

I think removing the use of the keyboard is probably the better solution if we want to keep supporting Linux. I'd like to keep the interface the same as on Windows and Mac. It might take me several weeks to get around to this however since it's a reasonable piece of work and I don't have a lot of free time at present.

[therealrobster]

Thank you again.

I can see why you want to do it the way you want to do it, it is more work, but it is a better result in the end.

Would the mouse still be usable to select the various areas of the screen? I imagined the keyboard would only be for pressing escape or similar? It's more just a question out of interest, as this is obviously beyond my capabilities.

However, I'll help however I can. Be it testing, any code that I can contribute (I am a hobbyist), or anything you can think of.

Just call me in and I'll do whatever I can.

Thank you so much. For me, Talon was missing the ability to be able to click on any part of the interface in any software, and I had no idea that your software existed. I'm really excited to get it set up.