search

twilio / twilio-video-processors.js

Twilio Video Processors is a collection of video processing tools which can be used with Twilio Video JavaScript SDK to apply transformations and filters to a video track.
Other
33 stars 21 forks source link

use TFJS instead of TFLite #43

Closed trickpattyFH20 closed 2 years ago

trickpattyFH20 commented 2 years ago

What is the feature that you would like to see? Please describe. Use the TFJS BodyPix model rather than the TFLite "MediaPipe Selfie Segmentation Landscape Model." TFJS Bodypix works in both Firefox and Chrome since it does not require Webassembly. https://github.com/tensorflow/tfjs-models/tree/master/body-pix

Is your feature request related to a problem? Please describe. Yes. Twilio video processors do not work outside of Chrome.

Describe alternatives you've considered Integrate TFJS BodyPix in a Twilio Video project outside of the Twilio Video Processors library.

Additional context I like Firefox.

alex-konoval commented 2 years ago

Twilio video processors requires OffscreenCanvas that is not supported by FireFox and Safari.

trickpattyFH20 commented 2 years ago

Could it be done with a regular canvas that is hidden and exists in the dom? @alex-konoval

alex-konoval commented 2 years ago

@trickpattyFH20 As I know OffscreenCanvas is used for performance reasons.

charliesantos commented 2 years ago

Hey everyone, thanks for submitting this ticket. We did not use tfjs and bodypix due to performance reasons - FPS and accuracy are significantly lower. Having said that, we are working on supporting other browser such as firefox and safari. Please continue to watch out for future releases.