search

openSUSE / zypper

World's most powerful command line package manager
http://en.opensuse.org/Portal:Zypper
Other
405 stars 110 forks source link

[feature request] decentralized package repository #453

Open 13werwolf13 opened 2 years ago

13werwolf13 commented 2 years ago

what I will write now will seem strange to oldfags. but I think that in today's world it might be a good idea. I’ll make a reservation right away that I offer an option and not a basic functionality. An option that will be disabled by default and enabled only by the user.

why not use the ability to preload packages for updates, as it is done in DC ++ and torrent networks. from users who have already downloaded these packages to themselves. the reality is that at home I have an Internet channel of one gigabit / second for ridiculous money, just like most of my friends. and mirrors can rarely give even 100 megabits / second. at the same time, in my local network I have as many as 5 devices with opensuse (server, router, computer, laptop, pinebookpro (the latter does not count because, due to its different architecture, it will not participate in this)). and at work in the server hundreds of servers. why do I need to milk one poor cow with packages with all this huge bunch of hungry mouths if you can optimize the software delivery process.

in addition, for the last couple of years, the local mirror (mirror.yandex.ru) has been constantly failing, it is impossible to use it on an ongoing basis. namely, it is selected automatically most often.

Of course, the question arises about the safety of this idea. but I see it as simple. the packages themselves can be taken from users in parts, and the metadata (including but not limited to their hashsums) via https from the main repository, which somehow guarantees that the package has not been changed.

an additional plus in preventive loading. when it comes time to update my device, since it works 24/7, it will be ready to install packages without waiting for download (users of countless texlive packages will definitely appreciate it), and a laptop that has not been turned on all day when turned on will receive the necessary packages faster on the local network than from a mirror , which at the same time will also unload the provider's channel.

P.S.: I'm sorry if I wrote something incomprehensibly, my level of English does not allow me to do it easily, I used an online translator.

mlandres commented 2 years ago

In fact we discussed this a couple of years ago. The major concern is not the integrity of the packages, it are legal implications when offering package uploads. That's why we do not offer any builtin solution, not even one that is turned off per default.

But it should be possible for you to mirror the repos or to host the local package cache on a distributed filesystem (like e.g. extremeFS). For zypp it needs to appear as a local directory, where it can read/write it's files. Admitted, our cache handling is not designed to be shared between different systesm, but this is something we can enhance.

13werwolf13 commented 2 years ago

In fact we discussed this a couple of years ago. The major concern is not the integrity of the packages, it are legal implications when offering package uploads. That's why we do not offer any builtin solution, not even one that is turned off per default.

sorry for a possibly stupid question, but still: what is the difference between asking for a license before downloading a package and asking for a license before installing it? after all, until it is installed, its presence in the system does not equate to "use". Or did I misunderstand the problem? in any case, you can also make a "black list" of packages that will not be automatically preloaded.

But it should be possible for you to mirror the repos or to host the local package cache on a distributed filesystem (like e.g. extremeFS). For zypp it needs to appear as a local directory, where it can read/write it's files. Admitted, our cache handling is not designed to be shared between different systesm, but this is something we can enhance.

I've tried using NFS as /var/cache/zypp and /var/cache/zypper, but this also leads to some problems, at least due to a different list of packages on different devices. in addition, sometimes when updating, for example, network-manager, the network turns off, which makes it impossible to continue updating, and keeping nfs constantly connected (especially on laptops with wi-fi) does not seem to me an attractive idea.

mlandres commented 2 years ago

The legal concerns apply to the torrent-like distribution. A client that offers packages for download must be sure the package license allows a re-distribution. Customers are afraid to (even accidentally) offer not-redistributable content or even private content. And so we are afraid as well. Zypp will not offer any local data for download.

If you offer local content on a distributed filesystem it's up to you to make sure the content is allowed to be distributed in that way.

With NFS it's probably better to mirror the repos and offer them via nfs:// to the clients. The client will then mount the repo vai NFS; download the packages in-advance to the local cache and install them for there.

13werwolf13 commented 2 years ago

The legal concerns apply to the torrent-like distribution. A client that offers packages for download must be sure the package license allows a re-distribution. Customers are afraid to (even accidentally) offer not-redistributable content or even private content. And so we are afraid as well. Zypp will not offer any local data for download.

If you offer local content on a distributed filesystem it's up to you to make sure the content is allowed to be distributed in that way.

I think I understand the problem. but what if we offer a torrent-like distribution only for local networks? as far as I understand, within my home network or my work network, I have that it is completely legal rights.

With NFS it's probably better to mirror the repos and offer them via nfs:// to the clients. The client will then mount the repo vai NFS; download the packages in-advance to the local cache and install them for there.

this method assumes the presence of a full mirror, this is an unaffordable luxury (in terms of the amount of space occupied on disks) for home, for work it is suitable for those who have the entire infrastructure on opensuse / suse, but it is also not suitable for those who have many different distributions in the infrastructure, regardless of the reasons. And if you use a local mirror, then the default http will work just like nfs.

RokeJulianLockhart commented 1 year ago

@13werwolf13, don't refer to people as “oldfags”. It lends no credence to what you state.

13werwolf13 commented 1 year ago

@13werwolf13, don't refer to people as “oldfags”. It lends no credence to what you state.

Sorry, I didn't mean to offend anyone. It's just an old silly habit since I was a kid.