PPSPP Interface

[ericxtang]

I think this implementation should adhere to a generic interface in Livepeer that handles broadcasting / subscribing to video streams - so we can swap out the underlining transport protocol.

Here is the proposal: https://github.com/livepeer/go-livepeer/pull/43/files

[ericxtang]

After talking about it more, it makes more sense for the interface to be generic (instead of Livepeer-specific). For integration with Livepeer, we can import go-PPSPP in Livepeer, implement a PPSPP_broadcaster and PPSPP_subscriber.

For PPSPP, we should have a constructor for creating a PPSPP node from a libp2p Swarm. The PPSPP node should allow data to be broadcasted into the node, and also to be received from the node.

[ericxtang]

This ticket is about defining an interface for this PPSPP package. Section 2 of the PPSPP RFC describes how the protocol works on a high level, and we can use that to inform our decision here.

We will use this interface to implement a PPSPP version of the Livepeer VideoNetwork, but we should keep the interface generic so anyone can use it as a reference implementation of PPSPP.

Note - there are 2 partial implementations of PPSPP: swirl and PYPPSPP that might be helpful as we are thinking about go-ppspp.

To start, I think the interface should allow anyone to:

• Start a PPSPP peer
• Start a new "swarm" with an ID (in Livepeer's case, that's a streamID)
• Join a swarm with an ID
• Write data into the swarm when you are the peer who started it.
• Read data out of the swarm (possibly via a channel or a callback function)

[derekchiang]

Looking at the code that we currently have (the Protocol API and the example), I think the Protocol interface already supports everything that you specified, with two caveats:

• We don't have a way to automatically discover peers. We'd have to connect peers manually like this. Presumably that's what we are trying to solve with #2 and #18.
• The interface doesn't allow for a good way to be "notified" when new data arrives, which makes reading data difficult (see how the example needs to sleep).

So in my mind, we have two paths forward:

• Add a notification-based API to Protocol for reading data. Then, we can simply implement the VideoNetwork interface with the Protocol API.
• Add an interface that's higher-level than Protocol. This new interface would probably be somewhat similar to VideoNetwork. The interface would be implemented with the Protocol API. Then, we'd implement VideoNetwork with this new high-level interface. Is this what you are suggesting?

I'm personally leaning towards the first approach as that would keep go-PPSPP a low-level, pure implementation of PPSPP, while minimizing the duplication between go-PPSPP and go-livepeer. However I don't have a strong opinion either way.

[sricketts]

I don't have a strong opinion here yet (just starting to think about it). But one thing to note is that @derekchiang's first proposal would be useful internally for testing go-PPSPP. Right now there are a handful of tests that use sleep to wait on messages.

So if we want to remove the non-determinism from those tests, we would want to build some kind of notification thingie anyway.

[dob]

I like the idea of PPSPP remaining pure, but keep in mind we'll likely be extending it to support things like webRTC transport anyway. Pure to me means that other PPSPP clients could potentially interact with clients running our version, but our version can potentially do more as well when speaking to other Livepeer clients. So, for example, extending protocol to support passing in a channel to be notified on received data seems reasonable.

Quick note on peer discovery, I think you already referenced two issues in which we shared some thoughts on it. My current thinking is that in the Livepeer protocol, as a subscribe request is routed towards the broadcaster, either the broadcaster passes a tracker back through the relay nodes to the subscriber which contains a peer list, or any intermediate relay node who already has their own peer list for this swarm could provide the tracker without going all the way upstream to the broadcaster. The node would take the tracker and join the swarm and make peer connections with some of the nodes in the tracker, and then proceed with whatever in-swarm peer discovery algorithm we decide to use.

Also take a look at this rfc which is the complement to PPSPP and describes the tracker protocol. I'm not familiar enough with it yet to know if we'd want to use it. I'd imagine maybe we'd borrow the format, but not the discovery mechanisms.

[derekchiang]

Had a chat with @ericxtang to scope out the "MVP" for this project. Basically we think there are two things left to be done:

• Add a more user-friendly API for reading data based on channels or callbacks.
• Define a Tracker interface for peer discovery. This project will have an in-memory implementation of Tracker for testing purposes.

We will keep this project minimal in that it will not be aware of Livepeer in any way. To interface with Livepeer, we will create a go-livepeer-ppspp project wherein we will have an implementation of the Tracker interface that finds peers via the broadcaster, and an implementation of VideoNetwork using this project.

P.S. @dob we might implement the tracker RFC at some point, but for the purpose of delivering an MVP we will probably go for something a lot simpler.