Feasibility Study: Minimum-Viable-Integration between `ethersphere/storage-incentives` and `livepeer/protocol`

[chrishobcroft]

Summary

For some time, it has been self-evident to me, that Livepeer and Swarm ought to work together more coherently.

This brief proposes a path towards an achievable minimum-viable-integration, to act as a proof-of-concept to build on in future.

Context

My self-evident position stems from considering the lifecycle for sharing video, which ultimately boils down to the following steps:

0. Capture - using camera and microphone, e.g.
   • using cameras / microphones in smartphones, or
   • professional digital cameras / dSLRs connected to computers / encoders
   • often involves some form of performance or expression.
1. Publish - sharing is caring, and can help us all be radically transparent FTW.
2. Transcode - re-size to make the content accessible
   • use source rendition (at e.g. 1080p or 4K) to generate "lighter" renditions (720p, 360p, 144p)
   • make it more "distributable" for slow connections, and more "playable" on weak devices.
3. Distribute - get the content to the people in the world who wish to watch live.
4. Archive - store source and transcoded renditions for watching later.

Now, breaking this down:

• [x] 0. expression and performance are abundant, smartphones with cameras / mic are ubiquitous, digital media is thriving.
• [x] 1. web apps such as justcast.it [1] are emerging, as ways of easily publishing content.
• [x] 2. Livepeer Public Network does this already, in sub-realtime, with accounting via Probabilistic Micropayments [2] and a Continuous Funding [3] protocol, both running on Arbitrum Mainnet.
• [ ] 3. This currently involves "centralised alternatives" (CDNs), with some decentralised options emerging ref: Media Network on Solana. However, a trustable soul told me Livepeer also have something which seeks to solve this. TBC
• [ ] 4. This also currently involves "centralised alternatives", with go-livepeer's object storage functionality able to plug in to third-party APIs. Some experiments have also happened with Filecoin [4], and livepeer.com continues to pioneer "Mint a Video NFT" functionality, using IPFS.

Both platforms run on EVM, with users transacting with 0x... addresses/keys, so both projects share a core foundation from a user's point-of-view.

With Swarm starting to provide realistic Ethereum-based alternatives to IPFS, with minimum-viable storage-incentives, the time feels right to start "leaning in" to working out how to integration these two "baby giants" of the Ethereum ecosystem can begin to act like siblings, instead of distant relatives.

But Chris, video is too heavy

In Prague, someone from the depths of the Swarm team told me that this kind of thing is not a good use case for Swarm: something about video being "too heavy".

I acknowledge this feedback, and agree that video is heavy. I am not defeated by it however, for reasons which I will attempt to explain now.

When livestreaming, it is possible to configure the bitrates and framesizes to be very very very small. For example, I can publish a source stream containing 50kbps of video data and 32kbps of audio data, totalling 82kbps. This might be at a framesize of 240p (426x240 pixels) and a frame-rate of 10fps, so the content may be slightly pixelated and even strobe-like. But it does work.

I can then use Livepeer Public Network to transcode this 240p source into a "lighter" 144p rendition, thus exercising step 2 in the process above. The streams then become available as a sequence of 2-second video segments, in 240p and 144p.

At this point, I would be looking to make a minimum-viable-integration, whereby an Orchestrator (livepeer/go-livepeer can push these 2-second video segments into Swarm. It would result in a maximum bitrate of ~100kbps, which I really hope Swarm is capable of handling.

Once we can prove this at a minimum-viable level, we can easily start to turn up the source bitrates, and observe where the bottlenecks occur, then fix them. Simples ;)

Implementation

So, Livepeer and Swarm have been talking for years, about how to work together. I haven't been involved in most of the conversations, but the tl;dr I get from them is not positive. Things appear too hard, or not priorities, or even things like "Swarm is on Gnosis" and "Livepeer is on Arbitrum" so bridging stuff is needed.

This, for me, is a great shame, and a huge waste of an opportunity to create a mechanism to make Livepeer a test-suite for Swarm. It also makes me wonder whether any evaluation of feasibility of integration is done at the client-level (go-ethereum <> bee), which provokes me to consider whether a more coherent integration could happen at the protocol-level (ethersphere/storage-incentives <> livepeer/protocol), which could help rationalise a tangible integration at the client level.

So, if it was possible, to connect the two things together somewhere in the stack, and begin to dream of being able to deliver a coherent use-case for these two Ethereum-based sibling-technologies, we not only help build out testing infrastructure for the Swarm project, but also we evolve a fully-decentralised live video publishing, distribution and archiving solution for the world.

It can be used for everything from sharing art and creativity and beauty and performance, through to "little brother is watching you"-style mobile-bodycam / dash-cam to create the data to allow things like community policing - all backed up to censor-resistant storage, and discoverable via ENS.

Conclusion

OK, I acknowledge that I don't know much about the storage-incentives behind Swarm, but I know a fair amount about the mechanics of Livepeer. Given the shared underlying foundations (EVM), and the shared Identity infrastructure (0x... public/private keypairs, ENS), it appears to me at least, to be a feasible approach.

But I'm confident someone with more awareness of the Swarm project can contribute something here, which will either debunk my wild and ambitious theories about how the world of media infrastructure could work, or will contribute to it by bringing additional facts to this conversation.

Either way, I hope the significance of a "joined up" solution to web3 video isn't lost on you. It can serve as some form of censor-resistant "audiovisual audit trail for humanity", where any device with a microphone (and camera) can publish live, with maximal accessibility, global distribution and permanent storage.

Soon enough, human souls would start to learn to only share the most beautiful and valuable content, to help them build their web3 reputations. And we might move towards a more beautiful, peaceful and creative world.

WDYT?

WDYT? Please leave a comment below.

References

[1] justcast.it is a web app which simply connects to your device's camera and microphone and allows you to start publishing video and audio to a livestream. Works on smartphones, laptops, desktops, and (probably) smart TVs with cameras and microphones. The project is by @victorges and source can be found here: https://github.com/victorges/justcast.it.

[2] Livepeer has a working implementation of Probabilistic Micropayments, integrated with the Livepeer Protocol. Primer is here: https://medium.com/livepeer-blog/a-primer-on-livepeers-probabilistic-micropayments-e16788b29331 Explainer is here: https://medium.com/livepeer-blog/streamflow-probabilistic-micropayments-f3a647672462, building on work by D. Wheeler and R.L. Rivest

[3] Livepeer has a working prototype of a "continuous funding" protocol, with mechanically controlled inflation distributed to participants. This has recently migrated to Arbitrum per this proposal, which was accepted by Livepeer Governance: https://forum.livepeer.org/t/a-proposed-scaling-strategy-for-the-livepeer-protocol/1583/4

[4] File.video was a product demo built to demonstrate a video hosting service powered by Filecoin and Livepeer. While the demo is no longer being maintained, the code remains open sourced. See file.video or https://github.com/livepeer/file-video

[IxaBrjnko]

I am working directly with both organizations and I know there is a lot of goodwill towards this development on both sides. I am not the only one on the Swarm side pursuing these goals, but for my part, I agree with the primary premise that "a more coherent integration could happen at the protocol-level" (perhaps most especially via the feeds mechanisms and the HLS standards in Livepeer). The needs beyond that technical integration extend to the idea of 'continuous funding' as Swarm does not necessarily provide 'permanent' storage, but must maintain the incentive structure to persist data, however there are clearly several ways to fractionalize and distribute the maintenance costs associated with both services across federated data layers in Swarm.

[jpritikin]

video is too heavy

This cannot be correct. A swarm based youtube-like site is approaching public alpha test.

[chrishobcroft]

@jpritikin wrote:

This cannot be correct.

Yeah, this is what I thought too. But the people I spoke to resisted strongly.

[chrishobcroft]

@IxaBrjnko wrote:

there are clearly several ways to fractionalize and distribute the maintenance costs associated with both services across federated data layers in Swarm.

Right now, solutions for such a mechanism to distribute the costs likely requires an imaginination for cross-network bridging solutions, as Swarm deploys on Gnosis Chain, and Livepeer on Arbitrum on Mainnet.

Perhaps a first step would be to deploy both protocols on the same public testnet. A coherent option would be to include both Swarm and Livepeer in the deployment of the Holešovice Testnet project from EthPrague (warn: this link is impermanent / web2), being targetted for EthBerlin3 in September 2022. Here is a link to the H project chat, which can be used 3 times, ping me if anyone needs more opportunities.

The specific asks of Swarm and Livepeer are to contribute automated deployment processes, for deploying their protocols onto this testnet. This would be a win-win-win with Swarm and Livepeer testing whether their stacks run on Verkle Trees, and developers being able to prototype integrations muuch more easily.

After this, a decision can be made about whether to add a network bridging solution, or to evolve towards a world where these two sibling protocols ought to live on the same network.

[significance]

@jpritikin wrote:

This cannot be correct.

Yeah, this is what I thought too. But the people I spoke to resisted strongly.

well, perhaps too heavy for swarm yet. but hopefully it will be soon!

for one-to-many broadcast (with a bit of latency) or archive playback the Swarm architecture should work well. range queries are supported through the Bee http endpoint.

awesome initiative @chrishobcroft - absolutely think a proof of concept is viable and very happy to support Livepeer and Swarm colab in any way i can 🤙

also - did i miss you in Prague? gutted! next time my friend! 😄

[IxaBrjnko]

hmmm... https://github.com/buidl-labs/Demux/tree/master/livepeerPull in https://github.com/livepeer/file-video