Chocobozzz / PeerTube

ActivityPub-federated video streaming platform using P2P directly in your web browser
https://joinpeertube.org/
GNU Affero General Public License v3.0
13.08k stars 1.51k forks source link

Allow trusted instances to provide redundancy for videos of unlisted videos #2719

Open tilllt opened 4 years ago

tilllt commented 4 years ago

I am running a main Peertube instance on my home server: lots of storage, bad bandwith. To mitigate for that, i am running a VPS instance with high bandwidth. I configured the Redundancy Settings to just mirror everything from the main instance. The problem is, that afaik unlisted and private videos are not considered for the redundancy. In a "hostile" federated environment, this is correct behaviour, but i think it would be great if Peertube could make a difference between trusted follower instance (i.e. run by the same admin) and untrusted instances, when it comes to redundancy mirroring.

If i have a unlisted video (personal, family stuff, stuff thats needs to be reviewed) - i'd like it to take advantage of my second VPS instance as well to provide higher bandwidth.

Chocobozzz commented 4 years ago

Private videos are not federated so you won't be able to add them in the redundancy system, and federating them changes the assumptions made on external videos' privacy.

tilllt commented 4 years ago

I don't know Peertube enough to have an expert opinion about what's the best way to achieve seeding of all videos of any visibility / privacy settings from multiple locations. My idea is that "trusted" instances, run by the same admin, could seed private and unlisted videos as well. What I DO know is that it would be desirable to seed also private and unlisted videos from several instances to overcome the problem of low bandwidth Peertube home servers.

JohnXLivingston commented 4 years ago

Why don't you invert your 2 servers? It seems more logical that the main server has a good bandwidth.

tilllt commented 4 years ago
tilllt commented 4 years ago

So IMHO a seeding "satellite" server makes much more sense. Think caching proxy.

JohnXLivingston commented 4 years ago
* less storage on vps

Ok, that's a good point.

* blazing fast uploads over local network

But the redundancy will take the same time... And.. Will be done multiple time if there are other instances that wants to redondate.

* might switch vps provider frequently to change to best offer

You can change provider, and change your dns records to keep your domain name pointing on the good vps.

* much beefier hardware on local server for transcoding

That's a good point. I think there is work in progress for remote transcoding.

tilllt commented 4 years ago
  • blazing fast uploads over local network
    
    But the redundancy will take the same time... And.. Will be done multiple time if there are other instances that wants to redondate.

The distribution process happens without any active participation, yes the transfer takes time, but the work of tagging, selecting a thumbnail, writing a description etc is finished and the video is published.

* might switch vps provider frequently to change to best offer

You can change provider, and change your dns records to keep your domain name pointing on the good vps.


No you cannot, there is no "good vps" just a good home server. So when changing the vps, you have to copy all the data, set up the new instance etc. So it's much more work then just setting the redundancy setting and following your main instance and wait for the videos to be copied over automatically...
JohnXLivingston commented 4 years ago

So it's much more work then just setting the redundancy setting and following your main instance and wait for the videos to be copied over automatically...

Are you keeping the same domain name when you change your server? If so, you will mess up the fediverse. If not, it will still have side effects on the fediverse.

On the other hand, you should have a backup strategy for your main instance. If so, it is easy to backup and restore when you want to change your server.

tilllt commented 4 years ago

The main (homeserver) instance is running on a ZFS filesystem with mirrored drives and a backup. So i think its pretty well secured. Maybe this is a misunderstanding, the VPS' i am talking about are 2-8€ a month low end VPS, which run Peertube instances just fine. Its not a nifty Hetzner Root server with remote ILO, for 100€ a month. so yes, having the main server at home is the easier and better solution (for me anyways).

why would i reuse the names of the "satellite" vps`they can be named 001.pixelscope.de 002.pixelscope.de etc. their name will never be posted anywhere, they just seed together with the main instance.

Ranlvor commented 4 years ago

Another use case for cache-only-instances for unlisted Videos: At my university some professors upload their lecture-videos to a centrally managed Peertube-instance. All videos are unlisted and organised in unlisted playlists. The students get the playlist-urls.

If this Peertube-instance get's popular it could get overloaded with video-traffic. So it would be nice to be able to scale it horizontally to more nodes if necessary. All nodes are managed by the same administrators.

c0xc commented 3 years ago

Similar question: Would it be possible to remove the origin instance from the list of peers when viewing through a mirror instance? https://github.com/Chocobozzz/PeerTube/issues/3512

jeena commented 2 days ago

Even without the home lab server if your audience is on different continents then having only one central server makes it so PeerTube is practically unusable on other continents because it's constantly buffering.

I too have mostly family videos marked as internal on my server in Germany where my own Family is. And it works great there.

But us and my wife's family live in South Korea and when we want to watch a video we need to download it first and then watch it locally because the PeerTube UI is constantly buffering. To watch a 2 minutes video it most of the time takes at least 5 minutes because of the buffering all the time. So people just complain to me that it's not working and then stop watching it.

This is because it's difficult for them to understand how to do the workaround with downloading the video first to your phone, then find it with a file-application and then watch it with your local video player.

Watching on the TV would be so nice but there also it's just buffering all the time.

I was looking into how to run two instances and doing geo DNS load ballancing so people in Asia would be able to download the videos from the server in Korea and the rest from Germany, but that's quite the enterprise way of doing it, very expersive, I can't justify that.

Running a second cache-only-instance works nice for public videos, I just wish it would also work for at least unlisted videos (which are also public per definition) but also internal and private videos.