HTML5 video streaming with collectionFS

[riaan53]

Hi,

Im busy with uploading videos, encoding them with ffmpeg to mp5, ogg and webm, and then serving them to the site again with a html5 video tag. I have been testing the whole time with small videos and did not notice the problem at first.

When I upload larger video, for example 94MB, a http get for the full file length will be called when using the url returned from fsFile.url() method. If you immediately click on the play button the whole movie will play. But if you wait for a while and then click play only the start of the movie will play and then no further chucks will be downloaded/streamed. This will result in a net::ERR_CONTENT_LENGTH_MISMATCH error because the length of the downloaded piece is not the expected length.

If you look at the header you can see the problem: accept-ranges:bytes content-length:97508214 content-range:bytes 0-97508213/97508214 content-type:video/mp4

It should be something like this (https://gist.github.com/paolorossi/1993068):

 var file = fs.createReadStream(path, {start: start, end: end}); //gridfs example - https://gist.github.com/psi-4ward/7099001
    res.writeHead(206, { 
'Content-Range': 'bytes ' + start + '-' + end + '/' + total, 
'Accept-Ranges': 'bytes', 
'Content-Length': chunksize, 
'Content-Type': 'video/mp4' });

Any suggestions or something I missed?

Thank you.

Regards, Riaan

[riaan53]

It looks like the browser made a range request and collectionFS responded with the range but with status code 200. It should be 206.

Status Code:200 OK
Request Headersview source
Accept:*/*
Accept-Encoding:identity;q=1, *;q=0
Accept-Language:en-US,en;q=0.8,af;q=0.6
Connection:keep-alive
Cookie:X-Auth-Token=Abqw1pwVspk_MvizskmlWX2o5hwMNabVnslSCTFsC9r; meteor_login_token=Abqw1pwVspk_MvizskmlWX2o5hwMNabVnslSCTFsC9r
Host:localhost:3000
Range:bytes=0-
Referer:http://localhost:3000/app/4MLzWJsyhXAdZLYtr
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
Query String Parametersview sourceview URL encoded
token:eyJhdXRoVG9rZW4iOiJBYnF3MXB3VnNwa19Ndml6c2ttbFdYMm81aHdNTmFiVm5zbFNDVEZzQzlyIn0=
store:mp4
Response Headersview source
accept-ranges:bytes
access-control-allow-headers:Content-Type
access-control-allow-methods:PUT
access-control-allow-origin:http://meteor.local
connection:keep-alive
content-disposition:inline
content-length:51381683
content-range:bytes 0-51381682/51381683
content-type:video/mp4
date:Mon, 12 Jan 2015 08:29:57 GMT
last-modified:Mon, 12 Jan 2015 08:26:08 GMT
vary:Accept-Encoding

[aldeed]

@raix recently refactored this code. Maybe there's a mistake.

Code in question is here if you want to submit a pull request.

[riaan53]

Hi @aldeed ,

I have sort of fixed the partial requests for gridfs but its still not complete. There is still an issue with the initial request that causes the video to not playback completely in chrome, but other than that range request works and can be tested with curl like this (just change the url):

curl --header "Range: bytes=0-20000" http://localhost:3000/gridfs/fs/451e22bce5cf6f36941158f0e2c4c2bb -o part1
curl --header "Range: bytes=20001-36907" http://localhost:3000/gridfs/fs/451e22bce5cf6f36941158f0e2c4c2bb -o part2
cat part1 part2 >> test1.avi

Or to request until the end of file use "Range: bytes=20001-"
Or request the full file "Range: bytes=0-"

Still need to do cleanup, comments, testing and change a few things as well. I first need to go on with some other things and will get back to this but anyone feel free to go ahead with the necessary changes for the PR if they want.

This fixes https://github.com/CollectionFS/Meteor-CollectionFS/issues/500 as well.

Regards, Riaan

[aldeed]

Great @riaan53! Looking forward to the PRs.

[AdamSutch]

Do any of you have a suggestion for a workaround or alternative package to use until this is fixed?

[riaan53]

Hi @AdamSutch ,

The reason for the above commits not working 100% is that the gridfs-stream npm package used by collectionfs uses old node style streams that does not have proper flow control. I have submitted a PR yesterday for gridfs-stream to upgrade to the new node style streams to be able to support flow control to be able to stream videos and improved uploads/downloads reliability and performance. And will submit a PR to meteor-file-collection hopefully today for byte serving (range requests), so in about a weeks time meteor-file-collection should support byte serving with flow control for video/audio streaming. I will try to get to a PR for collectionFS in as well after that. Im not aware of any currently available meteor package that support byte serving from gridfs (my goal is gridfs). Im sure the cloud storage packages like lepozepo:s3 will support it as amazon handels the requests.

Regards, Riaan

[gabrielstuff]

Hi @riaan53,

I encounter same issue and I'm not able to play video on iOS devices (iPad Air and iPhone 4S with iOS8) The site works well on chrome and firefox (webm format) but safari (desktop and iOS) fail to load anything.

It looks like you fix a lot of things. Could you share the branch / commit that we should use ?

Thanks

[gabrielstuff]

Please note, that for the sake of the video test, I stick to "video for everybody" http://camendesign.com/code/video_for_everybody/test.html and thus :

http://clips.vorwaerts-gmbh.de/big_buck_bunny.webm http://clips.vorwaerts-gmbh.de/big_buck_bunny.ogv http://clips.vorwaerts-gmbh.de/big_buck_bunny.mp4

Thanks

[gabrielstuff]

ok, I can confirm that for cfs:filesystem this two patches : https://github.com/riaan53/Meteor-http-methods/commit/382393195a77ec73cf34e1e5b1bec1f0a5c40edc and https://github.com/riaan53/Meteor-cfs-access-point/commit/f6d035ca30280f51851168f63b029b91b69f0334 on cfs:access-point and cfs:http-methods fix video playback on iOS and Safari Thanks @riaan53. Do you think you could update and bump version in order for everybody to be able to play video happily on their smartphones.

Thanks !!!

[riaan53]

Hi @gabrielstuff ,

Glad it works! Just a note: For video streaming (byte serving) to work on cfs-gridfs you have to upgrade to the latest gridfs-stream version https://github.com/aheckmann/gridfs-stream/pull/61 .

I still need to do some testing and cleanup before I can create the PRs and im bit side tracked on other project at the moment. Will try to get to it soon, but feel free to take it to a PR if you want :)

Just a note on testing - use larger videos so that you can skip to a piece that has not been downloaded yet so that you can see a new range request sent to the server.

Regards, Riaan

[gabrielstuff]

hey ! I'm using standard-filesystem not gridsf, so as for now it works. Does the http-methods and access-point need cleanups ? The addition seams clearly clear. But, I wondering how we can test that : /  

[riaan53]

Yip it looks pretty clear :)

If you are happy with the standard filestream I will upgrade the gidfs-stream package in cfs-gridfs and see if it works then I will submit the PRs.

[riaan53]

Hi @gabrielstuff ,

Unfortunately the upgrade for cfs-gridfs to use gridfs-stream v1.01 and mongodb to v2.0.16 did not went as smooth. Im getting a MongoError: no chunks found for file, possibly corrupt when the tempstore is trying to create a combined stream at https://github.com/CollectionFS/Meteor-cfs-tempstore/blob/master/tempStore.js#L364 .

Maybe some different in gridfs-stream as it was completely revamped for the new node style streams.

Do you think we must create the PRs for cfs:access-point and cfs:http-methods so long and then later fix the byte serving from gridfs?

@aldeed @raix any suggestions?

Regards, Riaan

[aldeed]

If there are some PRs we can review, then maybe we can better figure this out. Also note that we've merged most core cfs packages into a single repo now.

[ddresch]

Just for reference if somebody is searching for the response error which occurs if you download a video and also audio files with the current version (0.1.44).

HTTP 416 Requested Range Not Satisfiable

It took me a while to figure this out and I only changed the if-statement in access-point-handlers.js (line 66) to get it at least working again for cfs:filesystem.

if (range[1] != end || end == 0) end = fileSize - 1;

The patch by @riaan53 is probably the better solution. I only found this issue via searching for bytes=0-

[rootedsoftware]

I'm getting this same error when I try to load a page with an audio file in it.

Failed to load resource: the server responded with a status of 416 (Requested Range Not Satisfiable)

Then if I reload the error changes to

GET http://127.0.0.1:3000/cfs/files/recordings/5DZRswkSDZuoaHYPA/Voice0012.aac?…VG9rZW4iOiJ0eTdwR3cwRUFtUFJXNlFzbmstU2todGtMTWNqUDd4VnZIVTJ0Nkc1dFM4In0%3D 416 (Requested Range Not Satisfiable)

[gabrielstuff]

You can use our version on soixantecircuits It should fix it.

Le 11 sept. 2015 à 00:19, Josh notifications@github.com a écrit :

I'm getting this same error when I try to load a page with an audio file in it.

Failed to load resource: the server responded with a status of 416 (Requested Range Not Satisfiable)

Then if I reload the error changes to

GET http://127.0.0.1:3000/cfs/files/recordings/5DZRswkSDZuoaHYPA/Voice0012.aac?…VG9rZW4iOiJ0eTdwR3cwRUFtUFJXNlFzbmstU2todGtMTWNqUDd4VnZIVTJ0Nkc1dFM4In0%3D 416 (Requested Range Not Satisfiable)

— Reply to this email directly or view it on GitHub.

[rootedsoftware]

@gabrielstuff Can you post a link to yours? I don't see it in your repository.

[rootedsoftware]

@gabrielstuff I tried the repos you forked, but I still get the same error. Can you send me the link to the exact repos I'd need and their branch names?

Thanks

[mshamblott]

I'm having what I think is a related problem using cfs:gridfs trying to retrieve mp4 video files (mp3 audio works fine).
all packages updated. Chrome Version 46.0.2490.22 beta (64-bit)

For a collection holding both audio and video files, I have a helper with: return Media.find({_id:mediaId})

I have a video tag with: src="{{this.url}}"

When I hardwire in a URL for a video from the collection it plays fine. The problem seems to be no URL is returned to this.url: $("#myVideoDiv").html(); returns a nice looking video tag, just no src.

Is this still a known problem with the standard cfs code?

[gabrielstuff]

@c316 you should use :

• Meteor-cfs-access-point: https://github.com/soixantecircuits/Meteor-cfs-access-point and the MASTER branch
• Meteor-http-methods: https://github.com/soixantecircuits/Meteor-http-methods and the MASTER branch

@mshamblott unfortunately we did not work on the cfs:gridfs part which I think must have the same issue.

@raix and @aldeed does the bug has been addressed in recent release ?

Thanks !