Stream does not play correctly

[malamalca]

I am streaming raspicam via uv4l. The stream plays fine via Flutter Webview Component or Flutter VLC Player.

Trying to open strem with MJPEG widget produces half-rendered images, jittering, split images as seen in attached video.

Is it possible that --Boundary-- header preceding image produces streaming errors?

A link to a sample video and first part of my binary stream: video.mp4

[YowFung]

Hi @jaumard, it may be about gaplessPlayback property of Image Widget in your Mjpeg Widget. Set to true can solve it.

https://github.com/mylisabox/flutter_mjpeg/blob/01c3cd4c83544ee07feb4b37a188d00740e4267a/lib/src/mjpeg.dart#L98-L103

[malamalca]

I think the problem is the stream. It might be currupting data.

I captured binary stream inside updateStream() function and saved stream to file. First two or three images were ok, the rest of them were corrupted (partly rendered, vertically splitted,..).

After that I've captured stream with php (yes, I know..), saved stream to a file and extracted jpeg images. They were all ok.

[jaumard]

@malamalca when you captured the steam, did you capture the raw data coming out of the HTTP request or the data I parse ?

Because if it's the raw data coming out of the HTTP I'm really not sure what I can do about it... it would mean the the default dart HTTP package have issues. Or am I missing something ?

[malamalca]

Ok, I've solved my issue by converting playback from async to sync-ed code. There is a race condition in stream.listen which is adding data to _carry buffer.

gaplessPlayback prevents a bit of flickering too.

These are my edits. Would they be accepted as PR? https://github.com/malamalca/flutter_mjpeg/blob/cd4e921cc982152b645f55ea86090c1716eab8cf/lib/src/mjpeg.dart#L136-L206

[jaumard]

@malamalca please create a PR :) it will be easier for me know compare what you changed! But if it makes sense and fixes a bug, for sure I'll gladly merge your changes

[YowFung]

I also found relevant phenomena a few days ago: The MJPEG images can be displayed in macOS and iOS, but not in Android and Windows. I capture the binary data and save it as a JPG file, which has the same effect (macOS/iOS can, but Android/Windows can't).

Then I found that the binary data contained the HTTP protocol header. Like MJPEG frames, they all start with FF D8 ....

[YowFung]

In addition, there is one thing I don't quite understand.

https://github.com/mylisabox/flutter_mjpeg/blob/01c3cd4c83544ee07feb4b37a188d00740e4267a/lib/src/mjpeg.dart#L136-L143

Do invalid image frames appear to be used? I personally understand that it is correct to write like the following (I am not quite sure):

try {
    var isInvalid = false;
    await precacheImage(imageMemory, context, onError: (err, trace) {
        print("Invalid image: $err");
        isInvalid = true;
    });
    if (!isInvalid) {
        errorState.value = null;
        image.value = imageMemory;
    }
} catch (ex) {}

[jaumard]

That http header thing is weird! One things that I can do is if I have a second FFD8 I just ignore the previous one and restart. That will do the trick, but what I don't know is if a FFD8 can be part of an MJPEG frame, because if it's possible then it will not work at all as I'll reset the data each time...

Do invalid image frames appear to be used?

They don't, I mean as they are anyway passed to the image.value the package is trying to show them. That's maybe why you see some frame that are not right.

Not sure what's the best, let the corrupted images be shown or just ignore them (like you did with your changes).

[jaumard]

Pushed the fix as 2.0.1