Separate canvas returns black screen [Question]

[olsommer]

Hello community, @mozmorris , I got stuck and I really don't know how to solve it. Now I'm writing my question here because I am a little bit desperate. Sorry for that..

I am trying to use react-webcam as media input only and a separate canvas element as output. That's because I need to draw some things later on top.

But the only thing I see is a black screen.. although my console gives me some "positive" output, means that it logs readyState=4 and a video element).

I don't get it, it only renders a black screen.. Hope someone can give me a hint.

My question is a bit like #167

import { render } from "react-dom";
import Webcam from "react-webcam";

const VIDEO_CONSTRAINTS = {
  width: 640,
  height: 480,
  facingMode: "user",
  deviceId: "",
  frameRate: { max: 1, ideal: 1 },
};

const Stream = () => {
  const webcamRef = useRef() as React.MutableRefObject<Webcam>;
  const canvasRef = useRef() as React.MutableRefObject<HTMLCanvasElement>;
  const [mediaStreamReady, setMediaStreamReady] = useState(false);

  const draw = (ctx: CanvasRenderingContext2D, video: HTMLVideoElement) => {
    ctx.fillRect(0, 0, VIDEO_CONSTRAINTS.width, VIDEO_CONSTRAINTS.height);
    ctx.translate(VIDEO_CONSTRAINTS.width, 0);
    ctx.scale(-1, 1);
    ctx.clip();
    console.log(video);
    ctx.drawImage(
      video,
      0,
      0,
      VIDEO_CONSTRAINTS.width,
      VIDEO_CONSTRAINTS.height,
    );
  };

  const onUserMediaError = () => {
    console.log("ERROR in Camera!");
  };

  const onUserMedia = () => {
    console.log("onUserMedia: Camera loaded!");
    setMediaStreamReady(true);
  };

  useEffect(() => {
    const canvas = canvasRef.current;
    const ctx = canvas.getContext("2d");
    canvas.width = VIDEO_CONSTRAINTS.width;
    canvas.height = VIDEO_CONSTRAINTS.height;
    if (!ctx) return;
    if (!mediaStreamReady) return;
    const render = () => {
      const video = webcamRef.current.video;
      if (!video) return;
      console.log("render..");
      draw(ctx, video);
      requestAnimationFrame(render);
    };
    render();
  });

  return (
    <>
      <Webcam
        audio={false}
        ref={webcamRef}
        videoConstraints={VIDEO_CONSTRAINTS}
        onUserMediaError={onUserMediaError}
        onUserMedia={onUserMedia}
        style={{ opacity: 1 }}
        width={"200px"} // only to get safe that I receive a stream
        height={"100px"} // only to get safe that I receive a stream
      />
      <canvas ref={canvasRef} />
    </>
  );
};

export default Stream;```

[rvdende]

For anyone coming across this. You might be loading opencv differently, but it should be very similar no matter how you load it.

import { Button } from "~/components/Button";
import Webcam from "react-webcam";
import { cv, setupOpenCv } from "~/lib/opencv";
import { useRef, useState, useCallback } from "react";

let interval: NodeJS.Timeout;
export default function WebcamPage() {
  const webcamRef = useRef<Webcam>(null);

  const [capturing, setCapturing] = useState(false);
  const [cvready, setCvReady] = useState(false);

  const handleCanvas = useCallback(
    (canvas: HTMLElement) => {
      // console.log(canvas);

      if (cvready) {
        const src = cv.imread(canvas);
        const dst = new cv.Mat();
        // You can try more different parameters
        cv.threshold(src, dst, 177, 200, cv.THRESH_BINARY);
        cv.imshow("canvasOutput", dst);
        src.delete();
        dst.delete();
      }
    },
    [cvready],
  );

  const capture = useCallback(() => {
    interval = setInterval(() => {
      const canv = webcamRef?.current?.getCanvas();
      if (!canv) return;
      handleCanvas(canv);
    }, 50);
    setCapturing(true);
  }, [webcamRef, setCapturing, handleCanvas]);

  const stop = useCallback(() => {
    clearInterval(interval);
    setCapturing(false);
  }, [setCapturing]);

  return (
    <>
      <Button
        onClick={async () => {
          await setupOpenCv();
          setCvReady(true);
        }}
      >
        init
      </Button>
      {cvready ? "cv ready" : "cv not ready"}
      <Webcam audio={false} ref={webcamRef} />
      {capturing ? (
        <button onClick={stop}>Stop Capture</button>
      ) : (
        <button onClick={capture}>Start Capture</button>
      )}
      <canvas id="canvasOutput" />
    </>
  );
}