m3u8 playlist loads twice at start playing

[frutality]

This is probably a bug.

According to http://demos.flowplayer.org/basics/live.html we need to use flowplayer.min.js and flowplayer.hlsjs.min.js libraries. But if we do, after page load browser loads m3u8 playlist twice. In my case, this brought to situation I described here.

I investigated into this and found: first GET query of m3u8 file returns 200 OK, but second returns 304 Not modified. In this case, there was no playback. But if you go to another page and then click 'back' button, m3u8 file loads just once with 200 OK code. And stream is playing.

This happened in FF and Chrome browsers.

Fix 1: do not load flowplayer.hlsjs.min.js. In this case, stream just works in every modern browser. Fix 2: change webserver cache settings. Correct settings are etag off; and add_header Last-Modified ""; (on Nginx).

Question 0: m3u8 playlist loads twice — bug or feature? Question 1: which browsers support HLS live stream with flowplayer.hlsjs.min.js? Question 2: which browsers support HLS live stream without flowplayer.hlsjs.min.js? Question 3: what else reasons to use flowplayer.hlsjs.min.js?

Fiddle (just to see the source, it's not working because of CORS I suppose): https://jsfiddle.net/y1mhfsw8/2/

[phloxic]

Answering backwards: The fiddle cannot work because you're not loading jQuery which is required with video tag based setups: https://flowplayer.org/docs/setup.html#videotag-install

(these questions should be asked in our forum first, also docs are worth reading, at least https://flowplayer.org/docs/setup.html#live-streaming - and the references there) 0: I do not see the master playlist loaded twice in our demos, do you? 1: browsers featuring the MediaSource extension, see: https://flowplayer.org/docs/plugins.html#hlsjs-browser-support and the docs for http://demos.flowplayer.org/api/hlsjs.html - Safari 2: browsers/devices which do generic playback: most mobile devices, Mac OS Safari - and then browsers which have Flash enabled: https://flowplayer.org/docs/setup.html#flash-hls 3: among other things: a) better playback quality b) Flash now definitely dying, Chrome and Firefox will soon disable Flash by default c) manual quality selection: https://flowplayer.org/docs/plugins.html#hls-quality-selection

If you experience problems with your streams (but not with our examples), you should per https://flowplayer.org/docs/plugins.html#hlsjs-stream-compatibility verify whether the problem is present with the hls.js client itself in the hls.js demo: http://dailymotion.github.io/hls.js/demo/ - and for real debuggin access to the stream in question is required. From the network panel screenshot in your other report I suspect rather the cookie setting triggering the 2nd request, but that's a blind guess.

[phloxic]

Etag and Last-Modified per se can't be the problem:

$ curl -I 'http://d12zt1n3pd4xhr.cloudfront.net/ccc.m3u8' -H 'Accept-Encoding: gzip, deflate' -H 'Origin: http://demos.flowplayer.org'
HTTP/1.1 200 OK
Content-Type: application/x-mpegurl
Content-Length: 660
Connection: keep-alive
Date: Wed, 10 Aug 2016 08:16:51 GMT
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET
Access-Control-Max-Age: 3000
Last-Modified: Fri, 15 Apr 2016 00:25:10 GMT
ETag: "344627fe9d97cf557e87cb647c487605"
Accept-Ranges: bytes
Server: AmazonS3
Vary: Origin
Age: 473
X-Cache: Hit from cloudfront
Via: 1.1 326c3a58e0e7630f5d608316203a71a3.cloudfront.net (CloudFront)
X-Amz-Cf-Id: fnc8jsgWDvbcDnm6iFbvB2g9Qfa1J_3-hEiUkcPL4_l1GejYWObDrw==

works at e.g. http://demos.flowplayer.org/basics/long.html

[frutality]

First, thank you for answering!

Here is few screenshots where playlist is loading twice at demo page: http://imgur.com/a/iY9TK Please note, I don't know why, but after inserting link to my stream player gone maddness (temporarily) and loaded 4.ts and 5.ts files many times in a row. Much more than you can see on screenshot.

And please check if you see the stream http://scrumptious-duckbill-2524.vagrantshare.com/player/obs and if playlist is loading twice.

[phloxic]

thx for the minimal example, always very very helpful

Your stream is not spec compliant, TARGETDURATION must not exceed any segment duration:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:293
#EXT-X-TARGETDURATION:8
#EXTINF:8.333,
293.ts
#EXTINF:8.334,
294.ts
#EXTINF:8.333,
295.ts
#EXTINF:8.333,
296.ts
#EXTINF:8.334,
297.ts
#EXTINF:8.333,
298.ts

Any non-native implementation will have trouble with that.

At your sample page the DASH stream is redundant because a) you don't additionally load the dashjs plugin and b) even if you would load it, HLS (js) would be preferred where DASH can be played because because both rely on MSE (except Mac Safari where hls.js support is not stable enough yet, but there you also have the generic HLS alternative anyway).

[phloxic]

Actually the double loading happens most likely because you don't provide a master playlist (even with only one variant it's a good idea to provide a master with CODECS, BANDWIDTH, RESOLUTION etc. info - demo stream at http://demos.flowplayer.org/basics/live.html does exactly that). I get the same when I use a variant URL directly with any stream.

[frutality]

Thanks for help.

Actually I have not much control in playlist content. It's generated by nginx-rtmp-module. I changed some server configs, and now main playlist looks like this:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:PROGRAM-ID=1,CLOSED-CAPTIONS=NONE,BANDWIDTH=160000
obs_low/index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,CLOSED-CAPTIONS=NONE,BANDWIDTH=320000
obs_mid/index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,CLOSED-CAPTIONS=NONE,BANDWIDTH=640000
obs_hi/index.m3u8

So now it loads main playlist and then another, based on bandwidth I suppose.

I'll read more about nginx-rtmp-module settings, maybe I'll start some discussion at their Github. @sergey-dryabzhinsky told me dev branch has fixes for playlist generation, but I didn't try it yet.

[phloxic]

Well, in general HLS gets really interesting with adaptive bitrate switching (ABR) with several variants.

But what is the actual problem the m3u8 loaded twice?

[frutality]

If I understand question correctly, actual problem was when nginx sent Last-Modified and etag headers (this is turned on by default). When these headers was sent, first GET /hls/obs/index.m3u8 was 200 OK, but second GET /hls/obs/index.m3u8 was 304. Player told hlsjs: Video file not found. Playback never started.

And as I said, to fix this 'bug', I turned off Last-Modified and etag headers for m3u8 files.

Please check this if you're interested: http://ridiculous-ram-7183.vagrantshare.com/player/obs Check in Chrome browser. It can't play stream, but FF can at this moment. At the morning FF could't play it, so I think this is a bug, but hard to catch it.

[phloxic]

As shown in https://github.com/flowplayer/flowplayer-hlsjs/issues/41#issuecomment-238799241 it's not Last-Modified per se, but it is in your case. So Last-Modified is the symptom, but the cause is something else. My blind bet is on the cookie, or some other expiry-related thing you're doing on the server (FWIW at http://ridiculous-ram-7183.vagrantshare.com/player/obs session has expired since a while).

[phloxic]

@frutality - is your goal some kind of content protection?

[frutality]

No. My first goal is just working stream in all modern browsers. No business logic, no authorization, no content protection, even no transcoding. Just get input stream and publish it to everybody with HLS.

My goal is reached since I turned off Last-Modified and Etag. BTW, it's not only Last-Modified problem. If I turn off only LM, but not etag — problem persists (and vice versa).

Here is the new link: http://brilliant-fawn-8977.vagrantshare.com/player/obs Here is screenshot of my Chrome Network tab https://yadi.sk/i/dAPEV9J9u8qPx

[phloxic]

My bet is rather on the Cache-Control: no-cache (which one does usually for immediate expiration) and/or the 2 session cookies:

Anyway, it it works, don't fix it ;-) And, as for the bug title: the double request is to be expected with hls.js if there is no master playlist available.

[frutality]

I made a few experiments with Cache-Control as you advised here, but no luck. And I doubt cookies can be the problem, since.. this is just cookies, right? Two different cookies, set by standard Laravel installation.

Thanks anyway, maybe this information will be useful for somebody in future. Stream works now and I'm happy :)

For the bug title (and question 0) — this is totally feature, not a flowplayer bug.

[phloxic]

As the cookies are session related, in theory it may be interesting to know what exactly they do (I strongly presume it's session expiry related). As you can see in this demo https://blacktrash-org.prossl.de/fpssl/hls-crypt.php the way chosen in this case to protect the key is via a session cookie set on page load and an access expiry of 5 secs to the key on container click within the session while the key itself is served with a Cache-control: no-store header. Therefore I assume that the default interplay of Cache-control no-cache and Etag for all HLS assets caused the problem. - Anyhoo, glad you got it working.

[frutality]

Well, these cookies totally session related. Of Laravel docs:

Laravel also stores the CSRF token in a XSRF-TOKEN cookie. You can use the cookie value to set the X-XSRF-TOKEN request header. Some JavaScript frameworks, like Angular, do this automatically for you. It is unlikely that you will need to use this value manually.

As for laravel_session, from it's name we could guess it's just a session ID. It's not doing anything special I think.

[phloxic]

@frutality - this should now be working without any extra contortions as per https://github.com/dailymotion/hls.js/commit/5a6afe22462feee1f6dd8e1b2fb62dd828b5a184