Chocobozzz / PeerTube

ActivityPub-federated video streaming platform using P2P directly in your web browser
https://joinpeertube.org/
GNU Affero General Public License v3.0
12.78k stars 1.46k forks source link

IPFS to store videos #494

Open alxlg opened 6 years ago

alxlg commented 6 years ago

I think that what limits PeerTube adoption is that instances are perfect for personal/organization use but not to build a free service like YouTube where everyone can upload videos without limits. The issue is that storage has a cost and videos make the necessary storage grow quickly.

IPFS (InterPlanetary File System) can be used to solve the storage issue because every user can store the files by himself but it doesn't have a way to browse and interact with them. PeerTube instead has an awesome UI that can be used by everyone.

Would it be possible to combine PeerTube and IPFS? Ideally the instance administrator would limit the classic upload for each user but eventually let users upload videos by specify an IPFS address. I guess when a second user browse a PeerTube instance and want to watch a video hosted on IPFS, PeerTube provides it by reading from IPFS and not from its local storage. PeerTube instances would cache IPFS contents like IPFS users and admins would monitor IPFS cache impact on their storage. If a PeerTube user wants to be sure its video is available he just have to keep it on IPFS with his machine. This could have another advantage: if the used PeerTube instance won't be available anymore its users won't need to upload videos on other PeerTube instances if they are on IPFS: they would just "upload" the IPFS addresses.

I will be grateful to those who answer by denying or confirming my assumptions.

ROBERT-MCDOWELL commented 1 year ago

@Pantyhose-X IPFS (kubo) needs always the original file source (or an ipfs cluster source) to keep the chunks alive. if after some time (set from the gateways you are using) the source does not respond the chunks are deleted....

S7venLights commented 1 year ago

Would not a webRTC torrent app running in the users PC do the same thing when she/he leave it seeding the video? This also allows user to create "seedboxs" using RSS auto downloading torrent apps. Easy simple/KISS way of "distributing" the video hosting.

https://github.com/Openmedianetwork/visionOntv/wiki/Seedbox

I haven't read this whole thread but I had a similar thought to the one quoted above...

How to save PeerTube from constant dying instances

Many instances are constantly disappearing as they cant afford to maintain the server costs (Bandwidth). Torrents have existed for years and have kept most loved content up/available reliably. That's because every user is 'donating' storage and bandwidth. PeerTube only does this with people running the same video in a browser at the same time. I propose a PeerTube app that works essentially like any torrent app. It can cache a user chosen amount of gigs of video and seed it constantly.

In order for it to work well, it would help to have a more centralised instance, so more seeders exist, but I guess the seeding could just work across all instances. Additionally, if magnet links are used, a content creator could always seed their own content, ensuring it's still available even if it has low seeds or an instance goes down, since the PeerTube app could save the magnet links when someone subscribes to a channel.

As a result: Popular videos will always be bandwidth backed. And perhaps to solve storage issues, a content creator could opt to solely self host the video data on their machines/servers/seedbox but still have their channel hosted on the instance server. So in that case, the instance acts as the torrent website directory for the creator, but the data and bandwidth is run by creators and users. It would also reduce overall bandwidth use, since if users re-watch a video, it will still be in their seeding cache days later, so it could play locally and update the watch count in the instance. As to creators wanting to delete or update videos, they could do so in the app and that would signal the hosting instance to update the details and the instance can signal all other seeders to sync changes. Honestly this would be a win...

I can't take full credit for this Idea, it was partly inspired by https://autotube.app/ Additionally we could allow people to download, watch and seed videos with regular torrent clients, thus increasing seeders or seeding up-time.

Alternative idea: Use a similar setup to https://storj.io were people volunteer host data on a distributed P2P network with redundancy built in. (I suppose IPFS works similarly?) This way, even if a content creator went offline forever, their videos will still be seeding/hosted with other nodes.

Martinligabue commented 1 year ago

I see no one mentioned that Brave has built-in ipfs support, so every user coming from Brave (in addition to those who have the extension or ipfs installed) would contribute to the reseeding of the file. (This is without counting people who have the same video on their ipfs, even if they are not connected to peertube, since the hash is unique in the entire ipfs network)

ghost commented 1 year ago

I see no one mentioned that Brave has built-in ipfs support, so every user coming from Brave (in addition to those who have the extension or ipfs installed) would contribute to the reseeding of the file. (This is without counting people who have the same video on their ipfs, even if they are not connected to peertube, since the hash is unique in the entire ipfs network)

from the brave docs, looks like even if one does feel morally fine with a browser made by the Brave team it won't technically help at all by default, and with non default settings it's no better (at latency, dht discoverability, longevity) than any other instance of go-ipfs running on a random laptop:

"By default, Brave will load the URI being requested via a public HTTP gateway; however, it will also show an infobar that asks you if you’d like to use a local node to resolve IPFS URIs. If you choose to use a local node, Brave will automatically download go-ipfs as a component and will route future traffic through this node."

drzraf commented 8 months ago

Torrents have existed for years and have kept most loved content up/available reliably.

BT v2, 2017 attempted to somehow overcome this (with per-file hash trees and hybrid SHA256 computed metainfo files)

But IMHO a DHT (even more a media-related one) should be exclusively based on actual content's chunks of bytes IPFS does this (although in a slower, and maybe safer way than the KAD network)

Although aMule did most of this 15 years ago, crossing DHTs wasn't an easy task (hMule) back in in 2011 nor were 2013's Javascript implementations

I propose a PeerTube app that works essentially like any torrent app.

Now, in 2023, js-libp2p provides many useful bits of websockets, IPFS, KAD & Bittorrent, and it may sound possible to create a multi-DHT browser-based client (tightly connected to HTTP-service for metadata retrieval/mapping) and blur the lines between local and remote storage (or seeding both long-term local contents and medias being currently played online)

.... but the Bittorrent experience clearly shows that if moving/renaming a file stops participation, then it's harder for the network to keep up. A successful storage (and network) distribution depends on:

And for this very reason, even if the protocols/languages/libraries lowered the DHT barrier, browsers' hardened security would never allow fancy local filesystem operations. Since it's impossible to be too fancy within browser context:

drzraf commented 8 months ago

Digging further, it happens that the solution was nicely envisioned 8+ years ago and is called... LBRY (FAQ)

It's based on two distributed data stores (a blockchain and a Kademlia DHT) and a peer-to-peer protocol for exchanging data and brings the concept of content reflector

OrcVole commented 6 months ago

I think utilizing IPFS in Peertube would be a good idea. In case people are not aware of it, here is ipfs video:

https://ipfs.video/

One thing that could help would be for Peertube to calculate the IPFS CID for files it is making available as VOD. If the relevant CID were published alongside the file, that could help. Peertube wouldn't have to provide files using IPFS to do this. It would only need to calculate their CIDs and list them.