Add support for Mac OS X

[zjays]

From what I can tell, the add-on currently doesn't support media keys within Mac OS X (I tested it with Youtube on Firefox 39.0, running OS X 10.8.5).

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/25342256-add-support-for-mac-os-x?utm_campaign=plugin&utm_content=tracker%2F7600490&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F7600490&utm_medium=issues&utm_source=github).

[carlin-q-scott]

Thanks for letting us know about this. I'll try to look into it on my friend's Mac next week.

[zjays]

This might help: https://github.com/nightingale-media-player/nightingale-hacking/tree/sb-trunk-oldxul/extensions/apple-mediakeys http://wiki.getnightingale.com/doku.php?id=add-ons#apple_keyboard_media_key_support

It is an extension for Nightingale Media Player (XULRunner-based application) that allows the use of media keys on Mac OS X. The extension needs to be compiled with XCode.

[carlin-q-scott]

Thanks for the links; I will check them out, probably next week some time.

I tested the plugin on my friend's MacBook air and found that iTunes was hogging all the media key actions so that's probably why it's not working at all.

[Noitidart]

Here is how it was done with binary components: https://github.com/mstange/mediakeysappleremotesimfy/issues/1

That same issues shows how to do it with Objective-C, although this method may require accessibility. It is likely also possible to accomplish this with Carbon's RegisterEventHotKey. This Carbon routine has not been deprecated as no alternative exists. Here is a topic that shows how: http://stackoverflow.com/questions/4807319/register-hotkey

This is a great repo showing how to tap into CoreFoundation and Carbon routines: https://github.com/philikon/osxtypes

This is some great work!

[Noitidart]

Hey @carlin-q-scott I'm real excited to see OSX support for this, from my last post how feasible does this look? I love how you do all the work from ChromeWorkers!

[carlin-q-scott]

I looked at it a little bit and determined that it was too much of a commitment for me. If someone else such as yourself, @Noitidart wants to implement this I'd gladly accept the pull request.

[Noitidart]

Aw man, I have a lot of projects going on, and have requests from others to help them out. I can add this in line, and will work on it if I have to. It's very close to being done as you did excellent work on the Linux/Windows part.

[Noitidart]

Here's me setting up event tap on mouse from js-ctypes: https://github.com/Noitidart/MouseControl/blob/master/modules/workers/MMSyncWorker.js#L937-L1144

What I do know is that, the obj-c method cannot run from a chromeworker, that has to be on the mainthread.

What I did above seems to be exact same as what is being done here: https://github.com/mstange/SPMediaKeyTap/blob/master/SPMediaKeyTap.m#L75

The documentation on this says: https://developer.apple.com/library/mac/documentation/Carbon/Reference/QuartzEventServicesRef/index.html#//apple_ref/c/func/CGEventTapCreate

Discussion

Event taps receive key up and key down events if one of the following conditions is true:

The current process is running as the root user.

Access for assistive devices is enabled. In OS X v10.4, you can enable this feature using System Preferences, Universal Access panel, Keyboard view.

Firefox runs as root so we're gold here.

As it seems you will have to use CoreFoundation (with maybe mix of some objc) this will come in handy: https://github.com/philikon/osxtypes/

This screenshot procedure of mine uses CoreFoundation and Objc mix: https://github.com/Noitidart/NativeShot/blob/master/modules/workers/MainWorker.js#L1424

If you do get a chance to work on this before I do, if you need help setting up a mac vm i can help with setting up osx 10.10.1 on oracle virtualbox.

[carlin-q-scott]

Thanks for all the info @Noitidart and I understand that your time is also valuable. I considered setting up OSX on a VM but a friend of mine is willing to lend me his Macbook Air so that won't be necessary.

[Noitidart]

Wow thanks so much if you are able to do any of the OS X stuff - I really really appreciate it. I can very much help out. I can also make this a droppable into other peoples addons. :) Via npm and normal bootstrap.

[Noitidart]

I needed to add global hotkey support for an addon of mine. The hotkey i need, is if user hits the usual print screen button from anywhere it will pull up my screenshot addon. On Mac the combo for screenshot is Cmd + Shift + 3.

I made great progress on Mac. But I'm stuck I keep getting error on RegisterEventHotKey which means "invalid parameter(s)". I was wondering if you have some time it would be nice if you could put your eyes on this :)

https://github.com/Noitidart/NativeShot/blob/12b460cdf23236cc8ad1dcb762bba66cb0c634f5/modules/hotkey/HotkeyWorker.js#L290

[Noitidart]

Hey @carlin-q-scott I split it to another repository so its not cluttered by other code: https://github.com/Noitidart/System-Hotkey

The code we need to edit is in - https://github.com/Noitidart/System-Hotkey/blob/master/modules/hotkey/HotkeyWorker.js

To edit the types module that is in this repository - https://github.com/Noitidart/ostypes

Would be absolutely fantastic if you could help.

The linux vesion isn't working either, but I'm trying to use XCB for that, not DBus as you did in Media Keys.

Windows works great, I used your old technique. :)

I got a headache from debugging the Linux and OS X stuff for the past few days so posted for help - https://discourse.mozilla-community.org/t/system-wide-global-hotkey-help-on-xcb-linux-and-osx-carbon/7579

[Noitidart]

Wooohoo carlin here is mac support!! All thanks to @arai-a and @KenThomasses - So easy. But unfortunaately I couldn't get it to work from ChromeWorker, so this has to run on mainthread:

https://gist.github.com/Noitidart/7fd5569cd64a000e3027966a2ca90a36

The hotkey in that gist is command + shift + spacebar. The number 49 is key for spacebar. To get code for anything else, run this gist:

https://gist.github.com/Noitidart/8645b47b0e46a0eb284e

And then with firefox focused hit any key it will log it.

Super cool stuff!! :)

[gaelfoppolo]

Any news? @Noitidart @carlin-q-scott

[carlin-q-scott]

Somehow I missed Noitdart's last update. I can try finishing off his Gist sometime this week I think.

[Noitidart]

@carlin-q-scott That gist is complete. I am using it in my addon NativeShot too hook into printscrn across all systems (mac doesnt have print screen they use cmd + 3), check it out - https://github.com/Noitidart/NativeShot/blob/master/bootstrap.js#L3149

Fully functional.

[carlin-q-scott]

@Noitidart Well it's complete in that it registers a hotkey but I need to register a set of hotkeys and bind them to my app logic. I will also take a stab at using a ChromeWorker as I'd prefer it to run independent of the Firefox UI.

Thank you for sharing such complete code and a helper to figure out the key codes.

[Noitidart]

My pleasure! For Mac my tests showed it had to be on main thread. I asked around but didn't get a definitive answer. If you can get the mac portion to work from a ChromWorker that would superb! I would definitely change my method to yours!

On Mon, Jun 6, 2016 at 7:12 PM, Carlin Scott notifications@github.com wrote:

@Noitidart https://github.com/Noitidart Well it's complete in that it registers a hotkey but I need to register a set of hotkeys and bind them to my app logic. I will also take a stab at using a ChromeWorker as I'd prefer it to run independent of the Firefox UI.

Thank you for sharing such complete code and a helper to figure out the key codes.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/carlin-q-scott/browser-media-keys/issues/17#issuecomment-224145379, or mute the thread https://github.com/notifications/unsubscribe/AGE8iavnLchxYyKb7Q-IPrlAs6mvm4ROks5qJNOJgaJpZM4FlD8T .

[carlin-q-scott]

@Noitidart So media key events are not key events but system events which don't have the keyCode merthod but rather a keycode property. So when your code tries to get the keyCode it crashes Firefox. At least that's my interpretation of the Objective-C docs and this Swift code I found that handles media keys.

Here's my modifications to your keycode capture gist: https://gist.github.com/carlin-q-scott/5a5890a9f6d3fd7902164b44b10ab9a7. I would modified yours but thought you'd want to share it elsewhere without the change in the event type being captured.

I'm not finding the Objective-C docs terribly readable so I'm not sure how to modify the code that gets the keycode to use the new class member signature.

[Noitidart]

Wow very interesting find. Thanks Carlin.

So to get it to work for media key events, you just have to change NSKeyDownMask to NSSystemDefinedMask on this line:

var rez_add = objc_msgSend(NSEvent, addLocalMonitorForEventsMatchingMask, TYPES.NSEventMask(CONST.NSKeyDownMask), myBlock_c.address());

?

It looks like you are still accessing keyCode though. Or does the C have to be c?

In Objective-C if the selector has no colons (:) its a prop. If it has colons then its a method.

[Noitidart]

I linked your gist for media keys from my original gist, thanks brother this was some awesome team work:

 Note To use with Media keys change NSKeyDownMask to NSSystemDefinedMask thanks to @Carlin-Q-Scott - copy paste gist - https://gist.github.com/carlin-q-scott/5a5890a9f6d3fd7902164b44b10ab9a7

[carlin-q-scott]

@Noitidart Yeah, I figured out the NSSystemDefinedMask bit but getting the keyCode doesn't work, even with that slight casing change.

[Noitidart]

Try addGlobal instead of addLocalMonitorForEventsMatchingMask it's just a guess that system events may not be local.

[Noitidart]

This is an excellet article I think all we need is here - http://weblog.rogueamoeba.com/2007/09/29/

You need to check subtype too and ensure that it is 8 then it should have a keyCode for the media key but not in the keyCode field it will be in the data1 field.

    if( [event type] == NSSystemDefined && [event subtype] == 8 )
    {
        int keyCode = (([event data1] & 0xFFFF0000) >> 16);
        int keyFlags = ([event data1] & 0x0000FFFF);
        int keyState = (((keyFlags & 0xFF00) >> 8)) == 0xA;
        int keyRepeat = (keyFlags & 0x1);

This makes total sense, as it follows that only NSKeyEvents would have keyCode. So NSSystemDefined would be obtained via data1. Cool stuff.

This article also states a very important note:

One other thing to note is that these keys act as “global hot keys”, every application receives events for them, not merely just the foreground application. This whole public domain sample code is available in a file here.

Meaning we have no need for RegisterHotKeyEvent for a global, as by default these are global events. Very cool.

Honestly that swift stuff doesn't make sense too me. I haven't took the time to learn it yet.

[Noitidart]

Here are the media key magic numbers you need:

# hidsystem/ev_keymap.h
NX_KEYTYPE_SOUND_UP = 0
NX_KEYTYPE_SOUND_DOWN = 1
NX_KEYTYPE_PLAY = 16
NX_KEYTYPE_NEXT = 17
NX_KEYTYPE_PREVIOUS = 18
NX_KEYTYPE_FAST = 19
NX_KEYTYPE_REWIND = 20

[carlin-q-scott]

I spent hours trying to figure out how to de-refence the pointers to get the data1 value. I didn't realize that the cast function de-references pointers and that I had to access the value property on the result to get the native js value. I've updated the gist I posted with these changes so that it's printing out all the important values mentioned by @Noitidart.

Maybe i can wrap this up tomorrow but realistically this weekend.

[Noitidart]

This is looking like its coming out nicely! Awesome team work! Yeah the cast method is pretty cool but no tutorial is going to specifically tell you when you need to cast. It's just based on need based on what CData values we get in our ctypes (which ultimate depends on how we set up the types and how js-ctypes reads them as UInt64 or etc). That's why I console.log a lot of stuff.

In the code:

cEventType = ctypes.cast(cEventType, TYPES.NSUInteger).value;
var eventData = objc_msgSend(objc_arg1__aNSEventPtr, data1);

I think you should do a test on cEventType before trying to get the data1 field. As I'm not sure if all event types will have data1. If it doesn't, it will probably crash, or give a null pointer (0x0) but with objc my usual experience is it crashes.

[carlin-q-scott]

Haha, yes, it usually crashes if I do anything wrong :(.

In the actual code I'm only going to handle subtype 8 but I'm also trying to figure out if I can use that EventTypeSpec you used for the hotkeys to filter the events.

[Noitidart]

Superior work brother! If you can get media keys working with the carbon method (EventTypeSpec/RegisterHotKeyEvent) that would be ideal because this doesn't have the overhead of triggering for all the other keys/system events.

Re getting the objc method or the carbon method into ChromeWorker's - I'm not sure if either method would be easier. I was not able to get either method working from a worker. If you can pull that off that would rock!

[carlin-q-scott]

So it looks like iTunes gets the events as well, even when I return null from the callback. The official documentation implies that returning nil will kill the event chain but maybe iTunes is called before Firefox.

I've checked in what i have so far into my master branch.

[Noitidart]

The returning nil which is ctypes.voidptr_t(ctypes.UInt64(0)) will block the event from going into Firefox as we set up a local hook with addLocal.... This is for sure.

This is my theory: If you want to block it system wide you will have to use addGlobal.... Using that though MIGHT require user to enable assisstive/accessiblity devices/permission for Firefox. When using this with NSKeyMask you have to enable assitive for sure, as otherwise its a keylogger, i tested it. Hopefully it will work with NSSystemDefinedMask without assistive, because asking users to enable that is a headache and lots will fail at it.

[carlin-q-scott]

Oh, nil is zero... interesting...

Thanks for clarifying how Firefox does it but that makes me wonder why the Firefox hotkey are entirely broken for media keys.

The addGlobal documentation has this statement in place of the nil statement for addLocal:

You are unable to change the event, merely observe it.

And later in the same doc:

Events are delivered asynchronously to your app and you can only observe the event; you cannot modify or otherwise prevent the event from being delivered to its original target application.

I guess people could just clear their playlist and let iTunes receive the events. It's better than nothing!

[Noitidart]

Ooo totally makes sense for addGlobal to not allow blocking. Thanks for that.

Yep nil is 0 but because the obj_msgSend uses variadic, we should wrap it in a CType so we wrap it with voidptr_t otherwise firefox will throw an error.

How are the media keys broken in Firefox?

I read in a lot of places that the media keys were also intercepted by iTunes and it was a bother to everyone. They mentioned Spotify was able to get around it. I agree with you, for a v1.0 of this feature it's good to ask them to keep their iTunes playlist clear. In the mean time I'll add this research item to my research list (you know the list of ideas we have where we couldn't get it done right away but research each topic on the list for like 15min every day or so and eventually we get hit gold!)

How about the media keys on Google Chrome, I hear they allow their extensions to use it, if they are able to get around the iTunes issue then searching Chromium sourcecode would be a great place to check out.

[carlin-q-scott]

Yep, I'm working on double wrapping that 0 right now.

The Firefox hotkeys don't work at all, at least they weren't 6 months ago when I removed the claim that this add-on supported mac through the Firefox hotkeys after a bunch of people complained that they weren't working and my finding the same thing on my friend's MacBook air. At the time I assumed it was iTunes breaking Firefox since it reacted to key presses but maybe it was something else entirely.

I sort of have a list like that but it's for big ideas I'd like to pursue when I'm in between jobs. For small stuff I just throw it in JIRA and forget about it forever.

[Noitidart]

I sort of have a list like that but it's for big ideas I'd like to pursue when I'm in between jobs. For small stuff I just throw it in JIRA and forget about it forever.

HAHAHAHA!!! That is so funny!

Hey I came across something I'm trying an experiment with:

http://stackoverflow.com/a/11440924/1828637

In MouseControl I did a CoreGraphics (yes lol Carbon, Cocoa, and now third api set!) event tap for a mouse and this topic here seems like it said even taps work:

https://github.com/Noitidart/MouseControl/blob/master/modules/workers/MMSyncWorker.js#L1295

I know for keys I couldn't do an event tap without accessibility. I'll hack on this and keep you updated. Please keep going in the direction you are, please don't change it on account of this experiment, it might not yield any results, just wanted to give you a heads up.

[Noitidart]

Hey @carlin-q-scott can you please test this addon out on a mac. I have only a a vm so i dont have the media keys, but this was picking up mouse events. If it works it should tell you subtype of 8 in console, for media key, and for mouse it should be 7. If it is 8 it should tell you which media key you hit. I attached the xpi. Here is screenshot of it on my vm:

the xpi is attached as zip, please download and rename to xpi then install (github wouldnt allow xpi upload). or you can zip up and make xpi and install from the repository here - https://github.com/Noitidart/OSXMediaTap/tree/695e95b7809fec2ed67d1570de9d0813d3242d3e

OSXMediaTap.zip

[carlin-q-scott]

It picked up rewind correctly but then permanently crashed. Even restarting Firefox won't bring it back.

[Noitidart]

Oh sweet. Did it pick up play? You'll have to start firefox in safe mode and uninstall the addon, do you know how to do that?

I should have mentioned to load this as a temporary addon via about:debugging, so on restart it won't be there. We'll have to fine tune it here:

https://github.com/Noitidart/OSXMediaTap/blob/master/resources/scripts/MainWorker.js#L144

I think it crashed on that line. I don't know why it's not starting back up though.

[Noitidart]

Just updated the repo, the play key should work now. I was using wrong constnat for the play.

If it works. Please try to see if you can tweak it to avoid the crash, I can't simulate the media keys at all.

If you an succesfully avoid the crash, then try to discard the event by returning null.

If that all works, then this is the way to go. I'll have to write up the clean up procedure, as that is using a globally declared callback to avoid GC issues. It is also an endless (well it will be, for now it is just 30 seconds) event loop, so it locks the whole thread, so I'll have to give a kill switch from the mainthread so you can shutdown/terminate the worker. Doing worker.terminate will cause a bad crash as .terminate cancels any processes going on in the worker, but it doesn't do it properly for a running poll.

[carlin-q-scott]

I still can't get it to run again after the crash. Why was it important to remove the add-on in safe mode? I removed it normally and restarted Firefox. Using the about:debugging install option doesn't work at all for me. It didn't do anything after selecting the xpi file.

I'm going to sleep now so I'll resume this again tomorrow.

[Noitidart]

Oh I thought firefox was not starting back up after the crash. So firefox is starting, but the events are not coming in anymore? I think this is because I inserted a tap into the system. And it's still there or some wonky stuff even after firefox died. I'm sure wonky stuff is going on as I didn't do a clean up procedure yet. You'll probably have to restart your computer if the addon is running but the events are not coming in.

Loved collabing on this! See you tomorrow. We'll knock this out of the park soon enough!

Edit: Also I just verified, that after I get subtype 7 events, after the initial end of the loop (30sec), I don't get those events again until I restart the computer. This is for sure due to the no clean up.

[carlin-q-scott]

Oh, silly me. I opened the Browser Console the first time and then all subsequent times I was opening the Web Console. It detects the play button correctly but iTunes still opens.

[Noitidart]

Oh super sweet! So now can you please edit to return null when a media key is pressed. That should hopefully prevent iTunes but blocking that event.

[carlin-q-scott]

Yep, that worked. What's this magic you're using? CoreGraphics?

[Noitidart]

Woohoooo!!!! Without accessibility? If without, that is absolutely superb! I wasn't able to tap regular keys with this due to accessibility, but it makes sense the media keys are not apart of the regular keys. Can't really keylog the media keys ha!

Yep CoreGraphics (with objc mixed for extracting the data1 from event argument, we should try to find the CG way to do this as objc has overhead you want to avoid if possible).

Were you able to pin point the reason for crash?

Should I write up the clean up procedure?

Then should I write up as if you're going to use this or are you going to rewrite to fit into your addon? If you want to use this, I'll make it so that when you're done you just terminate MainWorker, and the MainWorker will terminate the PollWorker (which will be a subchild of MainWorker).

[carlin-q-scott]

No accessibility features required so we're good there.

I'd sure appreciate it if you could make your code more pluggable for this add-on and include a cleanup procedure. Is mainworker loaded into a page or chrome worker? I'm just unsure how I would terminate it from reading your code which is XPCOM based I'm guessing.

I think what I thought was a crash was just your 30 second loop terminating as it logs an error when it does at the very last line in the watchMediakeys method. I also found that the system locks the xpi file, even the one I uninstalled, but I suspect that would be fixed by a cleanup routine.

If it helps, I wrote a test case for pushing play but it requires the Firefox Add-On SDK to run and manually pressing the button. All it does is prompt you to push play and then makes sure the right data was sent from the key watcher.

[Noitidart]

Oo.

XPCOM is nasty, I avoid it at all costs. That is loading in a ChromeWorker. new Comm.server.worker spawns a worker lazily, you can supply just a path, I gave it some extra aruments to initiate it with some data. To terminate it, in bootstrap you just do Comm.unregAll('worker') and it will terminate the worker, and in the self.onclose of the ChromeWorker I'll have it stop the PollWorker loop, then terminate it by doing a Comm.unregAll('worker') from the worker.

I'm not sure how to write it pluggable. Right now you can import comm and ostypes submodule and then copy paste from OSXMedia addon. I'll take a look at your code though and see if I can do this for you. The interaction with your windows and dbus stuff confused me haha

You might want to use this chance and switch to XCB for Linux as well, because that uses the same technique as and the CG mac method, they spawn a child worker to spin a loop.

[carlin-q-scott]

I can see how some of the other key handlers could be confusing as they're faking running within a ChromeWorker except for windowsHotkeys.js (there's two different Windows implementations because neither works for 100% of people). Considering that windowsHotkeys runs in a ChromeWorker it should work as a good template for transitioning your code.

I'd like to tackle the Linux support some other time but XCB sure sound promising.

[Noitidart]

I looked at your code, I'm giving this a shot. Will keep you posted.

[RblSb]

@Noitidart thanks for support, but it seems it does not work in firefox nightly 49, the console does not give even errors (