Provide an option to enumerate all supported media types

[sole]

Right now the method isTypeSupported provides some amount of media support detection, but developers might not be able to come up with all the possible combinations of codec and rate (e.g. aType=video/webm; codecs="vp9" vs aType=video/mp4; codecs="avc1.4d4015"; width=426; height=240, etc

Could it be possible to provide a method that enumerates the available options, so that developers can provide that to the users?

This is an often requested feature when talking to developers about MediaRecorder

[yell0wd0g]

I think the original idea might have been to let the implementation decide which codec/container was best suited and hide all those details from the user. In actuality, however, the developer will very likely need to know/specify such info in order to know the file type generated and, moreover, different implementations of MR today support different configurations. With all that, enumerating the capabilities might be a very good idea.

I would also propose that they come ordered by the implementation by weak-preference order, IOW, the first ones in the list are the preferred ones, e.g. because there is support for accelerated encoding.

Something in the lines of static sequence<DOMString> getSupportedTypes ();, with each DOMString being a valid MIME type [HTML5].

Currently the API provides a static bool isTypeSupported (DOMString type);, I would say we should keep it for backwards compatibility and convenience.

[sole]

Yes! getSupportedTypes would be very nice - same as keeping isTypeSupported because it's more succinct than searching in the list returned by getSupportedTypes. Convenience!

[fluffy]

Often people want to be able to do something with the recording - thus why they made it. So they might need some control over what the format is. The browser can not figure out what is best with no hints.

[yell0wd0g]

@mreavy -- would you know someone in FF that could take a look at the PR #65 and have an opinion on it? I think this proposal has developer support and does not increase the fingerprinting surface, as I've argued in the said PR?

[yell0wd0g]

I think it's best to continue the conversation at https://github.com/WICG/media-capabilities/issues/28.

[npdoty]

WICG/media-capabilities#28 has been closed, but I don't see an explanation of the conclusion. Is this issue still relevant on this spec or is the discussion to be moved elsewhere?

I'm concerned with the increase in fingerprinting provided by a feature that enumerates every supported codec. I'm also concerned with the fingerprinting provided in all contexts for the boolean isTypeSupported method; the spec's first example shows how to enumerate the device's list of supported codecs without user permission or intervention.

[alvestrand]

The closing of the bug was done by a commit that added a MediaCapabilitiesInfo interface, which refers to a MediaEncodingConfiguration / MediaDecodingConfiguration. The current spec is here: https://wicg.github.io/media-capabilities/#decoding-encoding-capabilities

Not sure whether this answers the questions we started out with here; if it's supposed to be a solution, it needs to be incorporated by reference, I suppose.

[alvestrand]

I did leave https://github.com/WICG/media-capabilities/issues/68 after reading through the media-capabilities spec.

[npdoty]

Is the suggestion that MediaStream Recording is going to incorporate the Media Capabilities draft community group report to require implementation of the mediaCapabilities attribute? Or that MediaStream Recording will continue to define isTypeSupported and just that if per this issue you want an enumeration of every possible media capability, you should look to that other spec rather than MediaStream Recording?

If the suggestion is the latter, then I agree that we can close this issue and people who want that feature can continue that discussion in another place. I still believe that both isTypeSupported and especially mediaCapabilities as defined have substantial increases in fingerprinting surface without much justification.

[yell0wd0g]

@npdoty

In time, I'd like this Spec to be focused on (1) recording, and not on (2) enumerating capabilities, so when the Media Capabilities is ready, I'd be in favour of delegating (2) onto it.

Re. fingerprinting, I'd say it's better to discuss this in the context of Media Capabilities, since your concern would affect not just Media Recorder, but other encoding sources (and even decoding); i.e. MediaEncodingType is defined for both encoding and transmission, if you think there's a fingerprinting surface issue bc of the first, it would be reasonable to assume the second is affected as well, correct?

[npdoty]

@yellowdoge that makes sense. I'll open separate issues re: fingerprinting on Media Capabilities.

I understand wanting to delegate the capabilities functionality, which I think would be a good reason to close this issue. But as this spec is intended to go to CR in the short term (right?), we also need to review its current functionality, even if that functionality may shift depending on the subsequent development of other features.

[bradisbell]

I believe this is more of a MediaRecorder API issue. For the scope of this problem, it matters not what the device is capable of recording in. What's relevant is the type that was actually recorded.

If you use the MediaRecorder without specifying a MIME type, there is no way to determine what was returned to you, and thus no way to reliably set a subsequent Blob type or name a file via normal conventions. Short of digging into the bytes and sniffing the type for yourself, there's no way to know if you recorded a WebM, Ogg, or what codecs were used. You shouldn't have to set a type to know what was recorded. Furthermore, in cases where you do set the MIME when instantiating the MediaRecorder, it seems unreliable to assume that this setting is guaranteed to control the output.

[yell0wd0g]

@bradisbell that comment is IMHO tangential to the current bug: the ondataavailable blob has a type that will tell you what exactly is being captured. It was wired recently in Chrome (crbug.com/859199).