search

livepeer / go-livepeer

Official Go implementation of the Livepeer protocol
http://livepeer.org
MIT License
546 stars 171 forks source link

`current.m3u8` returns `ErrNotFound` (ever!) #1492

Closed chrishobcroft closed 2 years ago

chrishobcroft commented 4 years ago

Description Of Bug

When a Broadcaster is serving current.m3u8, it shows ErrNotFound when there are "current" streams available (and also when there are no "current" streams available).

Steps To Reproduce

Steps to reproduce the behavior:

  1. Start a B with ./livepeer -broadcaster -currentManifest

  2. Wait for ***Livepeer Running in Broadcaster Mode***

  3. Run curl http://localhost:8935/stream/current.m3u8

  4. See Unexpected Behaviour:

    ErrNotFound

  5. Start streaming into the B with stream key obs-studio

  6. Run curl http://localhost:8935/stream/current.m3u8

  7. See:

    #EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:PROGRAM-ID=0,BANDWIDTH=4000000,RESOLUTION=1920x1080
obs-studio/source.m3u8

  8. Run the following:

    ffmpeg -re -f lavfi -i \
   testsrc=size=500x500:rate=30,format=yuv420p \
   -f lavfi -i sine -c:v libx264 -b:v 1000k \
   -x264-params keyint=60 -c:a aac -f flv \
   rtmp://127.0.0.1:1935/test_source

  9. Run curl http://localhost:8935/stream/current.m3u8

  10. See Unexpected Behaviour:

    #EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:PROGRAM-ID=0,BANDWIDTH=4000000,RESOLUTION=500x500
test_source/source.m3u8

  11. Press ctrl-c to cancel the ffmpeg process started in step 8

  12. Run curl http://localhost:8935/stream/current.m3u8

  13. See Unexpected Behaviour:

    ErrNotFound

Expected Behavior

At step 4, even with no "current" streams, I would expect to see something that a player of .m3u8 file extensions may be able to recognise, like so:

#EXTM3U
#EXT-X-VERSION:3

This way, a player of .m3u8 extensions will at least be able to monitor that URL, and begin to play content it if content appears. For example, it's possible to add such a file to a playlist in VLC Player, and for it to start playing when content appears.

At step 10, I would expect to see both "current" streams shown, like so:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:PROGRAM-ID=0,BANDWIDTH=4000000,RESOLUTION=1920x1080
obs-studio/source.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=0,BANDWIDTH=4000000,RESOLUTION=500x500
test_source/source.m3u8

This way, some more "sophisticated" software for playback of content from m3u8 files can offer a User a choice of streams.

At step 13, I would expect to see the original stream (obs-studio) continue to be shown, as it is still "current", like so:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:PROGRAM-ID=0,BANDWIDTH=4000000,RESOLUTION=1920x1080
obs-studio/source.m3u8

This way, it is possible to see at least one "current" stream published on /stream/current.m3u8

Environment

image

j0sh commented 4 years ago

@chrishobcroft Thanks for the report and describing your use case. As you observed, the current.m3u8 endpoint only returns the most recent stream. If that stream goes away without another stream coming in, an error is returned from the endpoint.

The current.m3u8 endpoint is really a leftover from the v1 days when the stream name wasn't always easy to get, and it's kind of stuck around as a debugging helper. It isn't meant as a robust feature that's ready for production use.

At step 10, I would expect to see both "current" streams shown, like so:

Modifying the behavior so that it returns all the streams as part of a master playlist is an interesting idea, but would be a new feature, rather than being a bug in the current behavior.

To support that use case right now, a list of the streams currently active on the broadcaster can be obtained with /status endpoint on the CLI port. Eg, curl http://localhost:7935/status. The Manifests field can probably be transformed into something you're looking for. Full acknowledgement though, the /status endpoint is also pretty rough (there are a mix of broadcaster and orchestrator fields), and the usual caveats apply about exposing the CLI port.

I'm wondering if having an endpoint with a simple JSON list of active streams would be a more versatile approach, so you could adapt it to your use case, and others as well. Maybe one day we could have a GraphQL-like API. Eg, "give me a list of streams names" or "give me a list of all streams and their transcoded renditions" or "give me all 720p transcoded streams".

chrishobcroft commented 4 years ago

@j0sh thanks for the prompt response.

I'm wondering if having an endpoint with a simple JSON list of active streams would be a more versatile approach

I guess this is when it comes to a question of compatibility with existing and emerging browsing and playback software...

JSON is maybe good for raw metadata, but I would be interested in research on emerging standards in file extensions / wrappers for URLs specifically for media playback, perhaps even specifically for linear (not on-demand) channels.

Maybe there is some emerging area of open-source software for consuming multi-channel streams?

Or as I alluded earlier, perhaps the emerging M3U Playlist file format (source:wikipedia), specifically Extended M3U (source:wikipedia) and M3U8 (source:wikipedia), might actually be fit for purpose.

It gets mentioned online:

a) loosely in association with IPTV a lot, which seems to fit with Livepeer's livestreaming use case (if not the on-demand stuff!), and

b) around Apple's developer ecosystem, e.g. these Apple Developer articles adding_alternate_media_to_a_playlist and creating_a_master_playlist.

Interestingly for me, this might be something to include in the Integrate Livepeer with Apple. application, what do you think @nelsorya?

Incidentally, does anyone know why .m3u8 format was chosen in the first place in @livepeer @ericxtang @dob @yondonfu @jozanza maybe?

To support that use case right now, a list of the streams currently active on the broadcaster can be obtained with /status endpoint on the CLI port. Eg, curl http://localhost:7935/status.

Yes, and I just tried this, and it does show all "current" streams.

Which is very interesting if a user is operating on the same system as the B, and can somehow curate the content and re-stream locally.

I wonder about an integration of a local Broadcaster's "available streams" could be displayed in e.g. OBS Studio's "Scenes" selector. It might make for a very powerful UI for curation / live-mixing of livestreams / livestream production suite / director's chair.

Maybe one day we could have a GraphQL-like API. Eg, "give me a list of streams names" or "give me a list of all streams and their transcoded renditions" or "give me all 720p transcoded streams".

Yes, if it makes sense to put the content parsing on the broadcaster server side, why not? Would it require more ports to be opened in addition to 8935?

I'm I do also like the idea of blockchainifying it somehow, a la @graphprotocol, or even somehow integrating with @ensdomains for content labelling / metadata on @ethereum (a @golang project), and towards ETH 2.0 e.g. @prysmaticlabs (a @golang project).

This could also start to help link the currently livestreamed content to the back catalogue of recordings stored on @ipfs (a @golang project).

Modifying the behavior so that it returns all the streams as part of a master playlist is an interesting idea, but would be a new feature, rather than being a bug in the current behavior.

OK, makes sense, although in some ways, whether this is a bug or a new feature depends on what /current.m3u8 was intended to do. If it were called /latest.m3u8 then this is no bug :) labelling, huh :| !

I will leave this issue open, so that we can agree what the new feature should be, and then I will open a feature request, OK @j0sh?