Pefromance issue on makeImageSnapshot

[mahmoud2802]

hello

I want to scroll Text and makeImageSnapshot on every 40ms but when snapshot start screen getting lock and flickering. how can I do something like that?

const ref = useCanvasRef();

  useEffect(() => {  
      setInterval(async () => {
       const image = ref.current?.makeImageSnapshot();
        if (image) {
          // you can use image in an <Image> component
          // Or save to file using encodeToBytes -> Uint8Array
          //const bytes = image.encodeToBytes();
        }

      }, 40);    

  }, []);

  const x = useValue(0);
  const progress = useSharedValue(0);

  useEffect(() => {
    progress.value = withRepeat(withTiming(2, { duration: 1000 }), -1, true);
  }, [progress]);

  useSharedValueEffect(() => {
    x.current = mix(progress.value, 0, 100);
  }, progress); // you can pass other shared values as extra parameters

  return (
    <>
      <Canvas ref={ref} style={{height:200,width:400 }}>
        <Text
          x={x}
          y={100}
          text="Hello world"
          familyName="tahoma"
          size={70}
        />

      </Canvas>

    </>
  );

[wcandillon]

It may not be appropriate to take a snapshot every 40ms, what are you trying to achieve? Did you try something like react-native-view-shot?

[NikitaDudin]

@wcandillon It will be excellent to add readPixels with similar on WebGL2 readPixels behaviour. I think, just updating of Uint8Array should be work more faster. That will be helpful for creating textures as pixels array in real time with next applying that texture into WebGL context scene.

[wcandillon]

We can expose this function. Before we do, can you tell us a bit more about your use case?

[NikitaDudin]

I am planning to use Skia to compose images with filters, shaders and masks, geometric primitives on a canvas, then read as pixels and apply as webgl data texture to the model in real time. Current taking snapshot function slow for that(~20ms(taking snapshot)+~20ms(encode to base64) for canvas real resolution 1024x1024 and 20 elements on the canvas on iPhone 12). Maybe read pixels will be more faster.

[mahmoud2802]

@wcandillon I want built something like that .so i need to get array pixel of canvas every 40ms (25fps) and convert it to to my led scroller. I did it in native side android by moving text by animation and take a take snap shot of my view every 40ms and its work without any problem .but in react native i can't do it and performance doesn't allow me to do that.

[wcandillon]

Thanks for providing us with these use cases. This is very helpful. There are two things that currently make this operation slow for a real-time use case:

• The image encoding step. That could be improved by providing a Uint8Array to a method like readPixels(). Or even providing support for SkBitmap.
• Currently makeImageSnapshot is creating a cpu-backed surface, executing the drawing, and then giving us the result. This is quite slow and we're investigating our options there. There is also the canvas.readPixel method which we cannot support at the moment because of the way we execute the drawing commands on a separate thread.

We will investigate if we can provide makeImageSnapshot directly the gpu-backed surface. If we can, providing the necessary API to get the bitmap directly without encoding should be fairly easy.

[wcandillon]

@NikitaDudin @mahmoud2802 we're currently investigating this issue but it may take us a while to offer a good solution for this. In the meantime, I would like to explore workarounds that may fit your use cases. Here are two ideas on top of my head:

• One of the issue is that we redraw the picture on a cpu-backed surface, what if we would provide you with an API to draw directly to that cpu-backed surface? Then you would get the readPixel method that you require. This might be worth a try. Would that fit your use case potentially? (@chrfalch this would be canvaskit compatible)
• If you look at this example: https://github.com/Shopify/react-native-skia/blob/main/example/src/Examples/API/Freeze.tsx#L84 we have a way to convert a drawing to a shader but I guess you would still need a way to make this shader available to the outside world?

[wcandillon]

closing this issue as a duplicate of #700