johndbritton / teleport

Virtual KVM for macOS
GNU General Public License v2.0
791 stars 132 forks source link

Better Touch Tool Compatibility #20

Open daniloc opened 9 years ago

daniloc commented 9 years ago

Hiya!

I'm such a fan of Teleport—thanks so much for the years of love and maintenance you've shown it.

I've grown pretty dependent on BetterTouchTool. Among other things, I use three-finger clicks to close windows and save my hands a trip.

I've noticed that click events get eaten by Teleport on the local machine, but their gestures never reach BTT on the remote. Any advice on workarounds here, or pointers to the appropriate place in the source code to twiddle with?

I can see fixing it a naive way—optionally dropping local clicks with >n touches instead of forwarding them—or making Teleport (somehow) share events with BTT on the remote machine. Appreciate any insight you might share.

johndbritton commented 3 years ago

Thanks for reporting and apologies that you've never gotten a response.

I don't know exactly what's happening here, but I have a hunch that says the way events are being forwarded from one Mac to another causes them to be collected before they reach BTT on the source and executed after they would have been collected on the destination Mac.

I'm going to leave this open for now. If anyone else has had experience with BTT and Teleport please add your experience and reproduction steps.

sugitach commented 3 years ago

I also use BTT with teleport.

One characteristic behavior is that if the BTT on the host side has a chrome only setting, the behavior after conversion when chrome is the active window on the host side is sent to the client side. If chrome on the host side is not the active window, even if the same setting is made in BTT on the client side, it will not be reflected. I suspect this is happening because BTT processes and transforms events before teleport.

johndbritton commented 3 years ago

I suspect this is happening because BTT processes and transforms events before teleport.

My bet is that it's dependent on the order of launching the applications. Here are some steps you can follow to try things out:

  1. Quit Teleport and BTT
  2. Start BTT
  3. Start Teleport
  4. Observe behavior
  5. Quit Teleport and BTT
  6. Start Teleport
  7. Start BTT
  8. Observe behavior

I think the behavior will change as it may be dependent on the order of the apps creating event taps with macOS.

ElhemEnohpi commented 2 years ago

Hi, I'm new to Teleport, it seems very nice compared to some of the things I've been using until now. Thanks!

There's a feature request at the end of this, but tl;dr: please provide a way for Teleport to notify BTT when it's controlling the remote machine, either by coming to the foreground as an application, executing an applescript, or some other method. Ok, here is the long story:

My understanding is that BTT taps into the trackpad drivers at a fairly low level, and does all of its own gesture detection. If you choose "Show Live View" from the "View" menu, you can see what BTT sees. When using Teleport, the remote BTT doesn't see anything at all from the host - it's oblivious to any trackpad gestures. I don't know how Teleport works exactly, but if it sends continuous multi-touch coordinates to the remote (does it?), then the remote Teleport instance is somehow sending these to the OS in a way that BTT doesn't see. It would probably be possible for the BTT developer to hook into Teleport's trackpad data stream, but he may or may not be interested, depending how much work it is versus how much demand there is - especially if Universal Control works with BTT (which we have yet to find out).

On the other hand, Teleport does a good job of sending keyboard shortcuts generated by BTT on the host, through to BTT on the remote (unlike a surprising number of other KVM and similar apps, like Synergy/Barrier, and even Apple's own Screen Sharing). You can have BTT on the host convert any special trackpad gesture not already supported by Apple and Teleport, like a 4-finger tap, into a unique keyboard shortcut that no app uses, e.g. shift-command-control-option-F1. That will be sent through Teleport, intercepted by BTT on the remote, and can trigger whatever action you want. It can target only specific applications on the remote, using BTT's ability to change its rules according to what application is in the foreground.

There are two problems though, related to the fact that Teleport doesn't come to the foreground as an application on the local host, when remote control is active. In order to prevent the same action from happening on both the local machine and the remote, when controlling the remote, you couldn't have trackpad gestures that trigger any actions other than keyboard shortcuts, which are the only thing intercepted by Teleport. BTT can re-interpret its own gesture-generated keyboard shortcuts, so you could set up all the actions on the host to be triggered by those instead of directly by the gesture. That's extra work, because you need both a trackpad gesture and a keyboard shortcut for every action on the local machine. However, there's an even bigger problem that was mentioned above - many gestures will be specific to a certain application, for example, to close a tab in a web browser. If the browser is active on the host while the remote is being controlled, that command will be sent through Teleport to the remote, where some other application may be active instead of a browser.

To get around this, there needs to be an application that comes to the foreground on the host, whenever Teleport is controlling the remote machine. That way, BTT can change its rule set, and send the unique keyboard shortcuts instead of the actual actions, and not send any application-specific commands for whatever application happens to be in the foreground on the local machine. One way of doing this is to use a BTT gesture to move control to the remote, instead of dragging the cursor across, in combination with a "dummy" application.

I've set up my system with an application I created in Script Editor, which does nothing, and is saved as an application called "Dummy", with "stay open after run handler" checked. In BTT, I have an "all apps" four-finger click gesture that triggers two things - a keyboard shortcut for Teleport to move control to the remote, that I set up by clicking the invisible-unless-you-hover (please change that!) "options" button on the screen thumbnail of the remote machine, plus an action in BTT to show the "Dummy" application. Then I have a set of gestures in BTT, only for the Dummy application, to send the unique keyboard shortcuts. I've only just started using this, but it seems to work ok so far.

It would be easier if there was an option in Teleport for it to come to the foreground as an app, whenever it's controlling the remote, so that BTT could recognize it. Then I could just set up gestures and actions for that. If that's not possible, then a way for it to run a script, or trigger or notify another application, whenever it switches to controlling the remote. That way I could still simply move the pointer to the other screen, instead of having to use a special gesture. So that's my feature request!

johndbritton commented 2 years ago

but if it sends continuous multi-touch coordinates to the remote (does it?),

It uses an Event Tap: https://developer.apple.com/documentation/coregraphics/quartz_event_services

This is higher level than coordinates and works kind of like a pipe. When you are controlling a remote machine, Teleport creates an event tap which captures your keypresses and gestures and then sends them over to be emitted on the remote machine as events.

I don't know exactly how BTT works, but it's possible that it is creating an event tap as well and consuming events before they get to Teleport to send to your remote machine. I would try starting the applications in different order.

I will be happy to review a pull request that improves compatibility with BTT, but I will not be working on any features related to BTT as I don't use it.

Teleport does a good job of sending keyboard shortcuts generated by BTT on the host, through to BTT on the remote

This leads me to believe that my hypothesis is correct. If BTT emits the keyboard shortcuts from a tap that is before Teleport's tap then it makes sense that Teleport is able to transmit those keyboard shortcuts to the remote machine.

Teleport doesn't come to the foreground as an application on the local host, when remote control is active.

This is not a change that I would be willing to make. If Teleport were to take foreground on the host machine it would likely disrupt the workflow of users who don't expect that behavior.

then a way for it to run a script, or trigger or notify another application, whenever it switches to controlling the remote.

I'd welcome a PR that adds this feature and would be happy to review. My suggested approach would be to create an NSDistributedNotificationCenter and send a notification when control changes between machines. Then from any other application on your computer you can listen for the notification and react.

johndbritton commented 2 years ago

@ElhemEnohpi thanks for the detailed feedback!