Blank Canvas after calling applyBlur() in React

[djaffer]

I am trying to use this package in react? Is there anything else that has to be done other than what is in the document? The canvas is just blank after calling apply blur.

[akhil-rana]

Hi @djaffer Thanks for creating the issue.

Can you please share a code snippet or repo link for me to test and try..

I haven't tested specifically on react but on vannila JS or TS projects. If you have a code snippet I'll give it a go.

[akhil-rana]

Also I'm planning on making a react specific guide and changes soon. No ETA though.

[akhil-rana]

@djaffer Just a note.

segmentBackground() needs to be called before applying any kind of effect. If segment background runs successfully then applyBlur() should run without any issue as it's just a css filter on top of the segments.

[djaffer]

I had to pause and play again for it to take it into effect.

    segmentBackground(this.myVideoRef.current, this.myVideoCanvasRef.current);
      applyBlur(10);
      this.myVideoRef.current.pause();
  this.myVideoRef.current.play();

I do get this error often.

Couple of suggestions/Good to have.

1) Move media pipe to npm dependency so CSP issue is not there. it to bit 2.106577a1.chunk.js:2 Refused to load the script 'https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation@0.1/selfie_segmentation.js' because it violates the following Content Security Policy directive: "script-src-elem 'self' code.jquery.com https://google.github.io/mediapipe/getting_started/javascript

2) For image background would it be possible to rotate it in the library using package https://www.npmjs.com/package/@jimp/plugin-flip

[djaffer]

Great work there! The issue was you have an event handler on the play. The video is already playing when the background is added. I had to pause and play again for it to take it into effect.

segmentBackground(this.myVideoRef.current, this.myVideoCanvasRef.current);
applyBlur(10);
this.myVideoRef.current.pause();
this.myVideoRef.current.play();

I do get this error often.

Couple of suggestions/Good to have.

1) I am getting CSP issue. Would you like to move media pipe to npm dependency so CSP issue is not there. it to bit 2.106577a1.chunk.js:2 Refused to load the script 'https://cdn.jsdelivr.net/npm/@mediapipe/selfie_segmentation@0.1/selfie_segmentation.js' because it violates the following Content Security Policy directive: "script-src-elem 'self' code.jquery.com https://google.github.io/mediapipe/getting_started/javascript

2) For image background would it be possible to rotate it in the library using package https://www.npmjs.com/package/@jimp/plugin-flip.

[akhil-rana]

Okay got it. Must be how react handles few things. I'll work on this. Thanks for the feedback.

Regarding your suggestions:

1. I know i didn't add selfie segmentation as a npm dependency. This is because mediapipe is using obfuscated JavaScript in their code. So it wasn't compatible in certain situations(using vitejs) The only way to get it to work was adding the script into the index.html file manually from the cdn link. I've also created an issue regarding this here on vite.

2. If you just want to flip the image for the user then just use css transform: scaleX(-1); Though I might add this as customisable param in future.

[djaffer]

thanks. Got it. Was there any particular reason to use Vite rather react. Actually, I am using it in webrtc to transmit the video stream with background so I will have to create a new image flipped. This is ok I understand if you want to keep your library generic but I am not sure besides video call what other are major use.

[djaffer]

thanks. Got it. Was there any particular reason to use Vite rather react. Actually, I am using it in webrtc to transmit stream so I will have to create a new image that is flipped. This is ok I understand if you want to keep your library generic.

[akhil-rana]

Vite isn't a react replacement but a create-react-app replacement. It uses modern architecture to compile and run applications way faster when compared to CRA.

I've made it keeping mind webrtc and video calls only. The self image should be flipped to user in a video call I think, so I'm using just css transform to flip the whole canvas in my other project.

[djaffer]

thanks, but the issue on the opposite side the issue is to make it mirror mode.

[akhil-rana]

Okay. I'll add a new param to flip the background in next versions. Thanks for the feedback.

[akhil-rana]

So if it's okay then should I close this issue for now?

[djaffer]

yes I can close