About tensor dimensions and layout

[lum1n0us]

I am not quite familiar with openvino and general knowledge about ML. Please allow me to ask this here.

---

https://github.com/bytecodealliance/wasi-nn/blob/556890b121dd1171665d835aba4d04a7e29e37dc/rust/examples/classification-example/src/main.rs#L35

I noticed in wasmtime openvino implementation, let desc = TensorDesc::new(Layout::NHWC, &dimensions, precision);. The code is using "NHWC" layout and "dimension" of wasm tensor, which is in "NCHW"([1,3,224,224]）shape, together. I assume "openvino-rs" will do extra to convert tensor data (from "NCHW" to "NHWC").

Questions are:

• In classification-example case, why not create wasm tensor in "NHWC" directly. Then there is no need for data pre-processing("NCHW" -> "NHWC")?
• It seems users can create tensors in either "NHWC" and "NCHW" layout. How make provisions for matching input tensor and model input port requirement? I mean should we always assume users can satisfy the input requirement or backends in runtime has to do data pre-processing(like ppp in openvino)?

[abrown]

The OpenVINO pre-processing that you observed in Wasmtime was a hack that OpenVINO forced on us but as you note here, that shouldn't be necessary any more. The current design of wasi-nn asks users to set up their tensors in the exact format expected by the model/graph; wasi-nn is low-level in this sense. In the past we've discussed adding more pre-processing functionality to the wasi-nn API (and that avenue is still open if you want to propose something!) but for now we have found that adding all the image-related helper functions to the image2tensor crate is "good enough."