"Fingerprint" is unclear

[jussi-kalliokoski]

Originally reported on W3C Bugzilla ISSUE-19803 Wed, 31 Oct 2012 20:39:04 GMT Reported by Chris Wilson Assigned to Chris Wilson

The "should" description of fingerprint makes it unreliable, and it is not clear how it might fail.

[jussi-kalliokoski]

Original comment by Jussi Kalliokoski on W3C Bugzilla. Thu, 01 Nov 2012 10:04:45 GMT

(In reply to comment #0)

The "should" description of fingerprint makes it unreliable, and it is not clear how it might fail.

Yeah, at least AFAICT there's no way of actually guaranteeing that the reliability of the fingerprint. IIRC on Windows XP you don't even have access to the names of USB MIDI devices. And even when you do, it's really hard to determine that if the user has 2 pieces of the same MIDI device, which one is plugged. Hence it's not a hard requirement that the fingerprint stays the same for that certain port, because it might not be possible to achieve, but it's still better than nothing and very useful for example for DAW software, because it can be made to work most of the time.

Maybe we should add a note to the spec describing the rationale here?

[jussi-kalliokoski]

Original comment by Chris Wilson on W3C Bugzilla. Thu, 01 Nov 2012 17:42:41 GMT

I think most DAW software just uses the index offset and the device name to guess; that's all you get in Windows. This is why I'm suggesting maybe we should simply remove fingerprint.

[jussi-kalliokoski]

Original comment by Jussi Kalliokoski on W3C Bugzilla. Wed, 05 Dec 2012 10:13:12 GMT

(In reply to comment #2)

I think most DAW software just uses the index offset and the device name to guess; that's all you get in Windows. This is why I'm suggesting maybe we should simply remove fingerprint.

That's incorrect. Through the Windows' MIDI APIs, sure, that's all you get, but you can get more information about the device via other APIs. I don't think we should make our design choices based on what the Windows MIDI APIs (ie. the lowest common denominator) provide us, especially since there are ways to get around these limitations, they're just not very convenient. I don't want our API to be as annoying to use as the Windows MIDI APIs.

A native application designer can go and get this information about the devices themselves to improve the user experience, a web application designer can't. I think it's imperative that we provide features like this from the v1 to provide the developers with the tools to design the best user experience possible.

[jussi-kalliokoski]

Original comment by Chris Wilson on W3C Bugzilla. Wed, 05 Dec 2012 18:36:40 GMT

TODO: Detail how implementations should "fake" uniqueness when it's not available, and how developers should react to failures.

[jussi-kalliokoski]

Original comment by Chris Wilson on W3C Bugzilla. Wed, 05 Dec 2012 20:34:12 GMT

Resolved: keeping fingerprint, better describing matching. https://dvcs.w3.org/hg/audio/rev/f572fe073d58

[jussi-kalliokoski]

Original comment by Jussi Kalliokoski on W3C Bugzilla. Fri, 07 Dec 2012 20:19:46 GMT

(In reply to comment #5)

Resolved: keeping fingerprint, better describing matching. https://dvcs.w3.org/hg/audio/rev/f572fe073d58

The current wording doesn't quite communicate what I was proposing at the teleconf:

If the system detects (for example) two devices attached to the user's computer with the exact same pool of data used to generate the fingerprint, the second one's fingerprint generation would inject information (of the previous attempted fingerprint not being unique) to theentropy pool, and a third device would do the same, and so on, thus maintaining association unless the order of the devices changes.

This, I think, is the best guess you can get at uniquely identifying a device, and the user agent doesn't need to maintain or store any information about devices the user has had. A possible implementation in JavaScript that would catch the gist of that:

var devices = new Map()

function createFingerprintForDevice(device) { var fingerprint var poolExtra = ''

do { fingerprint = createUUID(device.entropyPool + poolExtra) poolExtra += '#existed' } while (devices.has(fingerprint))

devices.set(fingerprint, device)

return fingerprint }

Not sure how to word this in the spec. Would it be worth dictating this, regardless of how good the implementation is at producing unique fingerprints?

[jussi-kalliokoski]

Original comment by Chris Wilson on W3C Bugzilla. Fri, 07 Dec 2012 21:10:24 GMT

(In reply to comment #6)

Not sure how to word this in the spec. Would it be worth dictating this, regardless of how good the implementation is at producing unique fingerprints?

Hmm. Well, the problem is that you can't come up with an ideal identifier, right? If I record in the fingerprint "this is MOTU MIDI Express input port #1 of 16", and then when I go to recreate based on that fingerprint there are only 8 ports with that name/manufacturer/version, I don't know if #1 of 8 is the right port or not (I don't know if they disconnected the first interface/8 ports or the second 8). I couldn't think of an algorithm good enough to dictate; the best I could do would be "look for a name match, and match "#n of m ports with this name" - if the number of ports with this name ("m") has changed, just fail. That would take care of "unplugging my AKAI USB keyboard controller doesn't make my Launchpad assignments stop working," but the above case (I unplug one of my identically-named interfaces) would fail. What do you think of that? Your code example is missing too many function implementations for me to completely confirm, but I think that's essentially what it's doing?

[jussi-kalliokoski]

Original comment by Jussi Kalliokoski on W3C Bugzilla. Fri, 07 Dec 2012 22:52:59 GMT

(In reply to comment #7)

Your code example is missing too many function implementations for me to completely confirm, but I think that's essentially what it's doing?

Well, the function definitions don't really matter, it could be:

var createUUID = (str) => str

Port.prototype = { get entropyPool () { return [ this.name, this.manufacturer, this.version, this.type ].join(String.fromCharCode(0)) } }

Although that's not a very appealing unique identifier. The important part is that createUUID is deterministic. And oops, I meant 'port', not 'device in the the original example.

Anyway, that's not what I meant. In cases where the user has changed the port ordering and the ports can't be uniquely identified, things will go wrong, there's nothing that can be done to prevent this. But the algorithm I just typed gives correct behavior in for example this case:

Ports A and B have the same entropy pool. Port B gets a fingerprint that has something added to its entropy pool to indicate that it's natural fingerprint was taken. The application uses both A and B. The user disconnects port A, and its fingerprint gets freed, while port B still has the same fingerprint that was associated to it for the session. The user reconnects port A and here, magic happens, the algorithm tries if the natural fingerprint for the port is available, and it is because port B has a modified entropy pool, so the application can associate port A back to its rightful task, even though the port ordering has possibly been flipped. Of course, when the user restarts the application, the ports will be flipped but there's really nothing we can do about that. Sound good? I don't think it's possible to get a better fallback mechanism than this.

[jussi-kalliokoski]

Original comment by Jussi Kalliokoski on W3C Bugzilla. Fri, 07 Dec 2012 23:05:03 GMT

Actually, I just made an assumption that I'm not sure is valid with all platform MIDI APIs: if you unplug device at index 0 while the application is running, your reference to a device that was in index 1 still works. I can't test this right now, I only have 1 MIDI device home at the moment, does anyone know better?

[jussi-kalliokoski]

Original comment by Chris Wilson on W3C Bugzilla. Fri, 07 Dec 2012 23:07:10 GMT

(In reply to comment #8)

Ports A and B have the same entropy pool. Port B gets a fingerprint that has something added to its entropy pool to indicate that it's natural fingerprint was taken. The application uses both A and B. The user disconnects port A, and its fingerprint gets freed, while port B still has the same fingerprint that was associated to it for the session. The user reconnects port A and here, magic happens, the algorithm tries if the natural fingerprint for the port is available, and it is because port B has a modified entropy pool, so the application can associate port A back to its rightful task, even though the port ordering has possibly been flipped.

I don't see how you're going to maintain that port B - in the enumeration of MIDI ports coming form the underlying hardware, not in the fingerprint stored by the app in local storage - has a modified entropy pool. Every time you enumerate ports, you're going to start from zero with the fingerprint map - so if A was removed, B will be the "natural fingerprint available" port. Yes?

I think we can suggest matching based on the available info - port name, manufacturer, version, even index of same - but if one of those ports is missing (aka, to use your example, "there are not the same number of ports with that natural fingerprint"), then all bets are off, and the match should likely fail. At the very least, that lets the DAW grab the exception and implement their own hook. I think that's captured in the current text, but not as "MUST".

In short: I think we can say "deal with the case when the port name/etc. are unique" - we can even say "deal with the case when the port name/etc. are not unique, but the number of ports that match that doesn't change"; but if the number of ports that match that "natural UUID" is different, it's going to have to fail.

[jussi-kalliokoski]

Original comment by Chris Wilson on W3C Bugzilla. Fri, 07 Dec 2012 23:13:39 GMT

(In reply to comment #9)

Actually, I just made an assumption that I'm not sure is valid with all platform MIDI APIs: if you unplug device at index 0 while the application is running, your reference to a device that was in index 1 still works. I can't test this right now, I only have 1 MIDI device home at the moment, does anyone know better?

I know in Windows MIDI, once you've grabbed a reference to the device (via MidiInOpen/MidiInGetID/etc), you have an HMIDIIN or HMIDIOUT handle, which doesn't change (should continue to work) - but of course, if you enumerated the devices in a list via midiInGetNumDevs(), called midiInGetDevCaps() to get the name/etc. BUT NOT A HMIDIIN, THEN disconnected, the list indices are going to be off.

I think it's similar in CoreMIDI - it all depends when you call MIDIGetDevice().

[jussi-kalliokoski]

Original comment by Jussi Kalliokoski on W3C Bugzilla. Fri, 07 Dec 2012 23:42:48 GMT

(In reply to comment #10)

I don't see how you're going to maintain that port B - in the enumeration of MIDI ports coming form the underlying hardware, not in the fingerprint stored by the app in local storage - has a modified entropy pool. Every time you enumerate ports, you're going to start from zero with the fingerprint map - so if A was removed, B will be the "natural fingerprint available" port. Yes?

I need to actually try this to be sure, but for example the Windows MIDI API has the notion of device ID, which is actually a pointer to a HMIDIIN instance, and it seems that even if the port's index has changed, the ID stays the same, so you can actually see that port at index something has a pointer that's already associated to a fingerprint. But I need to figure out if this is true, and if something like this can be done on (at least) all desktop platforms. It would be silly if you can't detect whether two midi port instances are actually pointing to the same port.

[jussi-kalliokoski]

Original comment by Jussi Kalliokoski on W3C Bugzilla. Fri, 07 Dec 2012 23:44:47 GMT

(In reply to comment #12)

(In reply to comment #10)

I don't see how you're going to maintain that port B - in the enumeration of MIDI ports coming form the underlying hardware, not in the fingerprint stored by the app in local storage - has a modified entropy pool. Every time you enumerate ports, you're going to start from zero with the fingerprint map - so if A was removed, B will be the "natural fingerprint available" port. Yes?

I need to actually try this to be sure, but for example the Windows MIDI API has the notion of device ID, which is actually a pointer to a HMIDIIN instance, and it seems that even if the port's index has changed, the ID stays the same, so you can actually see that port at index something has a pointer that's already associated to a fingerprint. But I need to figure out if this is true, and if something like this can be done on (at least) all desktop platforms. It would be silly if you can't detect whether two midi port instances are actually pointing to the same port.

(In reply to comment #11)

(In reply to comment #9)

Actually, I just made an assumption that I'm not sure is valid with all platform MIDI APIs: if you unplug device at index 0 while the application is running, your reference to a device that was in index 1 still works. I can't test this right now, I only have 1 MIDI device home at the moment, does anyone know better?

I know in Windows MIDI, once you've grabbed a reference to the device (via MidiInOpen/MidiInGetID/etc), you have an HMIDIIN or HMIDIOUT handle, which doesn't change (should continue to work) - but of course, if you enumerated the devices in a list via midiInGetNumDevs(), called midiInGetDevCaps() to get the name/etc. BUT NOT A HMIDIIN, THEN disconnected, the list indices are going to be off.

I think it's similar in CoreMIDI - it all depends when you call MIDIGetDevice().

Yeah, I just tried with a virtual device and it works like this on Linux/ALSA as well.

[jussi-kalliokoski]

Original comment by Chris Wilson on W3C Bugzilla. Sat, 08 Dec 2012 00:46:58 GMT

(In reply to comment #12)

I need to actually try this to be sure, but for example the Windows MIDI API has the notion of device ID, which is actually a pointer to a HMIDIIN instance, and it seems that even if the port's index has changed, the ID stays the same, so you can actually see that port at index something has a pointer that's already associated to a fingerprint. But I need to figure out if this is true, and if something like this can be done on (at least) all desktop platforms.

Hmm, yes. Would have to see what happens with device IDs when doing the unplugging dance. I don't have a Windows instance handy or I'd test.

It would be silly if you can't detect whether two midi port instances are actually pointing to the same port.

Well, we don't currently have a way to do that; I don't think we should be saying that we'll return the same MIDIInput instance, for example, or people are going to step on their own onmessage handlers. Or we need to explicitly say that. Actually, we probably should explicitly say that either way. Does that make sense?

If we do make MIDIPorts have singular instances, I should point out that we would NOT be able to make the equality work across Worker boundaries, if we take that on. Not sure it really matters.

[jussi-kalliokoski]

Original comment by Jussi Kalliokoski on W3C Bugzilla. Sun, 09 Dec 2012 11:01:33 GMT

(In reply to comment #14)

Hmm, yes. Would have to see what happens with device IDs when doing the unplugging dance. I don't have a Windows instance handy or I'd test.

Ahh, I just tried it, and of course, I'd forgotten about this! On Windows, you can only have one handle per port, trying to get another will give you an error, so once you do a midiInOpen(), your HMIDIIN handle becomes an exclusive access to that device. What a PITA, but at least it makes my algorithm work on Windows.

Well, we don't currently have a way to do that; I don't think we should be saying that we'll return the same MIDIInput instance, for example, or people are going to step on their own onmessage handlers. Or we need to explicitly say that. Actually, we probably should explicitly say that either way. Does that make sense?

Oh nono, that's exactly what the fingerprint does, it allows you to see if two port instances are referring to the same port. But yes, equality between two instances of the port might have ugly consequences, and is not a good idea.

If we do make MIDIPorts have singular instances, I should point out that we would NOT be able to make the equality work across Worker boundaries, if we take that on. Not sure it really matters.

I think it's best that equality is measured by comparing the fingerprints rather than instances in this case.

[jussi-kalliokoski]

Original comment by Chris Wilson on W3C Bugzilla. Mon, 10 Dec 2012 17:54:37 GMT

(In reply to comment #15)

Tweaked the wording to try to make these goals more clear. https://dvcs.w3.org/hg/audio/rev/d61cc0adee7e.