search

WICG / shape-detection-api

Detection of shapes (faces, QR codes) in images
https://wicg.github.io/shape-detection-api
Other
306 stars 36 forks source link

Bounding box is insufficient for AR marker use case #17

Closed stuartpb closed 7 years ago

stuartpb commented 7 years ago

The spec as it is currently written uses a DOMRect (with x, y, width, and height properties) for describing the boundingBox of a detected QR code.

QR code detection needs to be exposed in terms of all 4 independent corners of the code (which will likely form a non-square quadrilateral, from which rotation and perspective can be determined) if a QR code is going to be recognized in the frame for overlay purposes (example: https://stuartpb.github.io/quirc.js/test_webrtc.html) and used as a marker (as described on http://www.multidots.com/augmented-reality/, a page linked in the current spec).

yell0wd0g commented 7 years ago

You're very right, and also, Mac and Android implementations support it. I propose returning the four corner points in a clockwise fashion in addition to the bounding box, wdyt?

yell0wd0g commented 7 years ago

@stuartpb please take a look at https://wicg.github.io/shape-detection-api/#barcode-detection-api and see if it does the trick, thanks!

stuartpb commented 7 years ago

That's OK (if there's no existing structure in a pre-existing spec for defining arbitrary quads), but now I'm not satisfied with rawValue being a DOMString. QR codes can encode any kind of data (I'm using them for the bytes of a UUID, for instance): I'm not convinced a DOMString is the right way to expose their value.

On the other hand, QR codes can encode digits or alphanumeric content in a non-byte-oriented way, so exposing it as a DOMString may be the only sensible output; for use cases that encode it wouldn't be the first time a DOMString has been misused to encode arbitrary bytes as code points. We'd just need a note that UTF-8 interpretation of the string's otherwise-interpreted-as-Latin-1 codes would need to be achieved through decodeURIComponent(escape(rawData)).

yell0wd0g commented 7 years ago

rawValue is a DOMString which is a UTF-16, so I think it can represent every conceivable characters, but indeed the Wikipedia entry mentions symbols > 8bits. Since the OS APIs we're targetting here work with Strings, I think it's sensible to assume that they'll unpack the symbols into the UTF-16 alphabet. I'll add the note you propose as a caveat.