Can I use tensor as input in my browser?

[huanyufuchen]

Can I use tensor as input in my browser? I'm using offScreenCanvas as the input to detect singleface in another thread. But the browser I need to use doesn't support offScreenCanvas. So I want to use typearray that passes in the image directly

[vladmandic]

you can't pass a tensor as reference to a web worker as it's not a static type, but you can with ImageData and then construct a tensor inside worker from that. In your case, if you already have a typed array, pass that and again just construct tensor inside worker.

be careful regarding RGBA vs RGB as ImageData is typically RGBA and TFJS works with RGB inputs, so you might need to strip alpha channel unless you've already done that in your typed array.

Just a quick example:

  const tensor = tf.tidy(() => { // create tensor from image data
    const data = tf.tensor(Array.from(imageData.data), [imageData.height, imageData.width, 4], 'int32'); // create rgba image tensor from flat array and flip to height x width
    const channels = tf.split(data, 4, 2); // split rgba to channels
    const rgb = tf.stack([channels[0], channels[1], channels[2]], 2); // stack channels back to rgb
    const reshape = tf.reshape(rgb, [1, imageData.height, imageData.width, 3]); // move extra dim from the end of tensor and use it as batch number instead
    return reshape;
  });