Open ccarse opened 4 years ago
I was able to monkey patch this by doing this.videoRef.captureStream = () => stream;
I think it would be nice if instead of passing the model a video or image element we could pass a MediaStream. That way I could just give it the stream that is returned from navigator.mediaDevices.getUserMedia.
@ccarse - Thanks so much for this investigation! This is definitely something we need to keep in mind + something to add to the browser testing todos to make sure we can support the modern browsers. Let's keep this issue open as a note.
Thanks for following up!
Hello, any update on this topic ? I am having the same issue on Safari when using Yolo with mp4 files
This issue is caused by the Video
utility which we use in YOLO
, MobileNet
, and StyleTransfer
.
The Video
utility copies the user's video into a new <video>
object capturing the video stream -- which is not supported by Safari.
https://github.com/ml5js/ml5-library/blob/c3123cac0b1dfa0ed8e3e2588e8dea72ccd05aa8/src/utils/Video.js#L58-L63
I believe that the reason for copying the video is so that we can resize it into the shape that is required by the model, without messing up the video that is displayed on the page.
Funny thing is...it does not actually resize the input data! We are setting the width
and height
properties on the video, but the TensorFlow tf.browser.fromPixels
function gets the size from the videoWidth
and videoHeight
properties, which contain the intrinsic size of the video. These are read-only properties which we cannot set.
My recommendation is that we read the video at its current size and convert the current frame into a TensorFlow tensor first. Then we can use TensorFlow functions like tf.image.resizeBilinear
to resize the tensor of pixel data.
Dear ml5 community,
I'm submitting a new issue. Please see the details below.
β Step 1: Describe the issue π
It looks like captureStream isn't supported in Safari? Is there an alternative api I can use?
Here's my code: