Missing broader media use cases, such as slideshows, presenting tools, etc.

[tobie]

Seems use cases such as slide shows would also benefit from being controlled by remotes/media controllers/etc.

The current, audio/video element constraint means such apps will have to rely on hacks to gain access to these controls, such as looping over a silent audio file. This seems subpar.

[foolip]

What are the buttons on those remotes and how do people expect them to work? I've never used one.

[tobie]

For example, the Apple remote.

[jernoble]

Why wouldn't remote control events just be mapped to key events?

[foolip]

How is this remote treated in OS X? Is that play/pause button indistinguishable from the similar button on a MacBook keyboard, or can apps do something different with it?

[richtr]

It's worth noting that Keynote on iOS supports remote controls and it allows a user to control slideshows from a remote device. The technology being used for that is WiFi and Bluetooth. A Bluetooth API is currently being defined elsewhere.

I think such a use case would be in scope of the Bluetooth work. As you said, the current proposal would allow you to control slideshows but via an audio focus hack. Despite its technical impurity that could be made to work just as you would expect for slideshows and also reflects the way you would need to implement this use case in native applications today.

[richtr]

Also worth noting is that if you display the presentation on a second screen via the Presentation API then you can effectively control it from your device. Presentations is in fact one of the primary use cases for that API and is included in that specification.

[richtr]

Based on the stated use cases for the Presentation API covering this usage and also based on the possibility that the Bluetooth API could be used for remote control interaction, I propose we close out this issue for now and mark these use cases as out of scope of this work.

[jernoble]

@foolip

How is this remote treated in OS X? Is that play/pause button indistinguishable from the similar button on a MacBook keyboard, or can apps do something different with it?

There is no general mechanism for accepting remote control commands on OS X (though there are ways). Per-app support for the remote is implemented at a system level. However, if you'd like to see a general purpose remote control mechanism for third-party app use (say, Opera), then please file a Radar requesting such.

EDIT: And yes, the remote control and the hardware playback buttons use the same mechanism.

[tobie]

@richtr For info, on a Mac, both Keynote and iPhoto can be controlled using the dedicated hardware playback buttons, the remote just acts as a more convenient way to trigger the same actions in certain settings. Developers shouldn't need to be aware of implementation details of the remote control (does it use Bluetooth, Wifi, IR) in order to make it work, nor should they need to know whether the command to stop or play a slideshow was triggered by a remote control or the keyboard playback buttons.

I also don't think devs should be forced to come up with crazy hacks (silent audio files) to make things work on the Web platform, as that contributes to give the Web bad rep.

[jernoble]

@toble What exactly is the use case for using the keyboard media controls to control a slideshow that isn't satisfied by the arrow controls on the same keyboard?

[foolip]

@jernoble, of much greater interest to me than remotes on OS X are the keyboard back/play/pause/forward buttons. Is there a public API for registering to receive these and (e.g.) take audio focus from iTunes? From what little I've read it seems like all apps receive events and I'm not sure what coordination between apps there can be.

If an API for this does not already exist, and you think it's reasonable, I'd be happy to file a Radar issue.

[jernoble]

@foolip No such API exists. And I think it's absolutely worth a Radar. (Especially since I'd like to make use of it in WebKit as well.)

[tobie]

@jernoble:

What exactly is the use case for using the keyboard media controls to control a slideshow that isn't satisfied by the arrow controls on the same keyboard?

UX consistency. Why would users need to be aware that different media require different keyboard controls? You want to pause a movie? "Pause" button is going to work by default. Want to pause a slideshow? Same key no longer works.

[tobie]

@foolip related work around and radar bug report opened by Spotify in 2011. Also see GitHub project.

Keynote is able to steal media focus from iTunes, and apparently so is Spotify using above mentioned technique.

[jernoble]

@toble Please do not use the current implementation of the keyboard media controls in OS X (and hacks therein) to justify web standards. We do not want to codify the current behavior into the web platform.

[jernoble]

@tobie

UX consistency. Why would users need to be aware that different media require different keyboard controls?

UX consistency is not a use case.

[tobie]

@jernoble

Please do not use the current implementation of the keyboard media controls in OS X (and hacks therein) to justify web standards. We do not want to codify the current behavior into the web platform.

Why not? Paving the cow path generally isn't such a bad idea. What's wrong with the current behavior?

[jernoble]

@tobie

Why not? Paving the cow path generally isn't such a bad idea. What's wrong with the current behavior?

The GitHub repository you linked to is one example: the only way for a third party to get access to these key events is to install a window server event filter at such a low level that leaving it running causes some other applications to break. The media key commands work in Keynote but not Powerpoint. They work in iTunes, but not QuickTime Player. They work in Spotify (through a crazy hack), but only by breaking Adobe Fireworks.

There is no organizing principle here to be derived.

[tobie]

Well, it seems this just means the current implementation is crappy (which is kind of standard for cow paths, tbh). Or are you suggesting there are technical challenges that would make that difficult to implement on certain platforms (which I'd find… surprising)?

[jernoble]

There is no technical problem here. Delivering key events is pretty well understood at this point. But since the hardware media keys on OS X only support about five apps, you cannot use this one implementation to derive a design to be used by every webpage on every browser on every platform.

[tobie]

UX consistency. Why would users need to be aware that different media require different keyboard controls?

UX consistency is not a use case.

Heh. It's an important design principle, though. My main point here is that users should be able to interact with any kind of media in a consistant way however said media is implemented. And developers should be provided with the necessary primitives to enable this.

[tobie]

But since the hardware media keys on OS X only support about five apps, you cannot use this one implementation to derive a design to be used by every webpage on every browser on every platform.

I beg to differ. The fact that it's actively used by apps in production and that very successful apps have gone out of their way to enable this feature for their app shows that controlling apps with media keys is a highly desirable feature. But that's a discussion to have with the whatwg/media-keys folks.

[foolip]

@jernoble, does http://www.openradar.me/8817368 have all the detail required? Could you poke around a bit internally to see if there's any chance of adding an API for this? Otherwise getting this stuff working in Mac desktop browsers sounds like it's going to be a mess...