webmachinelearning / model-loader

🧪 Model Loader API
https://webmachinelearning.github.io/model-loader/
Other
30 stars 10 forks source link

Make MLTensor strongly typed? #39

Open wacky6 opened 2 years ago

wacky6 commented 2 years ago

Some issues with using a dict:

Discovered this during WebML prototyping for https://crbug.com/1338067

See related issue in WebNN https://github.com/webmachinelearning/webnn/issues/275

wacky6 commented 2 years ago

Alternatively, should we simplify to drop dimensions and simply use bytes[]?

https://www.w3.org/TR/webnn/#typedefdef-mlnamedarraybufferviews

ATM, I don't think we support arbitrary size inputs (i.e. CNN style network still has a fixed input / output size). Validation can simply check byte length matches the model's input/output description.

If the user provides an array with the correct byte length, but in a different type (e.g. uint16 -> float16), they just get wrong results.

huningxin commented 2 years ago

@wacky6

Alternatively, should we simplify to drop dimensions and simply use bytes[]?

Do you mean using ArrayBuffer instead?

wacky6 commented 2 years ago

I mean align with WebNN and use ArrayBufferView.

Since we just care about byteLength matches the expected input/output type+shape (both are known at graph build / model loading time)

huningxin commented 2 years ago

I mean align with WebNN and use ArrayBufferView.

Sounds good to me.

My open is whether it should support a union of ArrayBuffer and ArrayBufferView that is BufferSource typedef of Web IDL.

Although the WebNN prototyping CL 4006509 uses this typedef as workaround for a Blink generator issue, it reminds me whether we should consider that.