search

webmachinelearning / webnn-samples

🧠✨ Web Neural Network API samples
https://webmachinelearning.github.io/webnn-samples/
Apache License 2.0
108 stars 28 forks source link

Run semantic segmentation sample on full GPU pipeline #124

Open huningxin opened 2 years ago

huningxin commented 2 years ago

This is required by https://github.com/webmachinelearning/webnn/issues/226

To implement this pipeline, the semantic segmentation sample probably could execute following steps:

  1. Use mediacapture-transform to capture the VideoFrame
  2. Import the VideoFrame into a WebGPU texture (GPUExternalTexture)
  3. Execute WebGPU shader to do pre-processing and convert the GPUExternalTexture to a GPUBuffer
  4. Compute the MLGraph with converted GPUBuffer and the output is another GPUBuffer
  5. Execute WebGPU shader to do post-processing on the output GPUBuffer and render the result to a canvas.
  6. Create a VideoFrame from the CanvasImageSource and enqueue to the mediacapture-transform API's controller.
huningxin commented 2 years ago

Related to https://github.com/webmachinelearning/webnn-polyfill/issues/156

huningxin commented 2 years ago

depends on https://github.com/w3c/webcodecs/pull/412