Manjaro mirror?

[thibaultmol]

Hi,

I use Manjaro but love IPFS and was wondering if it'd be possible to use this for a Manjaro install. From my understanding Manjaro uses the same packages as the Arch repo (+ a couple special ones like the Hardware detection packages) . And since IPFS just uses each files' hash, I assume that the users who are hosting the arch mirrors would could also be hosting packages for Manjaro users?

Curious to hear what you think of this

Cheers,

[guysv]

It's gonna be a bit difficult because you need to point pacman to an ipfs gateway, without accidentally downloading the repository dbs (i.e. the repository index, which you download with pacman -Sy)

Another pitfall is that packages that you need but are not actually on IPFS gonna take a long time to time-out, which isn't fun, and you will need to go to your traditional mirror eventually. (although if you do download the repository dbs, you could know ahead of time what's available)

tl;dr using the IPFS mirror in manjaro is possible, but it needs some careful tweaks.

I guess the better approach would be to raise a separate manjaro mirror on IPFS, just be sure that packages common to both arch/manjaro get the same hash so we could share seeds cross-mirrors :)

[thibaultmol]

I have space and some bandwidth to help host such a mirror node. But would like someone like you who actually knows what they're doing when it comes to more complex IPFS stuff like this to handle the actual setting up of those systems itself

On Tue, 8 Dec 2020 at 09:41, Guy Sviry notifications@github.com wrote:

It's gonna be a bit difficult because you need to point pacman to an ipfs gateway, without accidentally downloading the repository dbs (i.e. the repository index, which you downloaded with pacman -Sy)

Another pitfall is that packages that you need but are not actually on IPFS gonna take a long time to time-out, which isn't fun, and you will need to go to your traditional repository eventually. (although if you do download the repository dbs, you could know ahead of time what's available)

tl;dr using the IPFS mirror in manjaro is possible, but it needs some careful tweaks. I guess the better approach would be to raise a separate manjaro mirror on IPFS, just be sure that packages common to both arch/manjaro get the same hash so we could share seeds cross-mirrors :)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/RubenKelevra/pacman.store/issues/37#issuecomment-740473609, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHWUMXCGZZ63P3ENLZDNHLSTXRDXANCNFSM4URBNBPA .

[guysv]

Hey @thibaultmol just wanted to say I got myself a nice VPS to run IPFS on, gonna look into mirroring manjaro this weekend hopefully.

@RubenKelevra could you please share what settings do you add archlinux packages to IPFS with? I tried to reproduce this cluster package hashes on my own and failed.

E.G

$ md5sum acl-2.2.53-3-x86_64.pkg.tar.zst
3a9b685c9b40591b71495748a058df2d  acl-2.2.53-3-x86_64.pkg.tar.zst
$ ipfs cat /ipns/x86-64.archlinux.pkg.pacman.store/core/acl-2.2.53-3-x86_64.pkg.tar.zst | md5sum
3a9b685c9b40591b71495748a058df2d  -
$ ipfs resolve /ipns/x86-64.archlinux.pkg.pacman.store/core/acl-2.2.53-3-x86_64.pkg.tar.zst
/ipfs/bafykbzacecx44p2x4tqkd7bsys6vwr2hbaii4vi664j6tv2gi355drxbg6222
$ ipfs add --pin=false --chunker buzhash --hash blake2b-256 --cid-version 1 -q ./acl-2.2.53-3-x86_64.pkg.tar.zst
bafk2bzacebc2pfhf7atdt6i2j4wnw3u5pky2545hqke7vz7fc3inpso5gkune

I took those add params from here, please tell me how to get the right hash :)

[guysv]

my bad

$ ipfs add --pin=false --chunker buzhash --hash blake2b-256 --cid-version 1 -q ./acl-2.2.53-3-x86_64.pkg.tar.zst
bafk2bzacebc2pfhf7atdt6i2j4wnw3u5pky2545hqke7vz7fc3inpso5gkune
$ ipfs resolve /ipns/x86-64.archlinux.pkg.pacman.store/core/acl-2.2.53-3-x86_64.pkg.tar.zst
/ipfs/bafykbzacecx44p2x4tqkd7bsys6vwr2hbaii4vi664j6tv2gi355drxbg6222
$ ipfs object links /ipfs/bafykbzacecx44p2x4tqkd7bsys6vwr2hbaii4vi664j6tv2gi355drxbg6222
bafk2bzacebc2pfhf7atdt6i2j4wnw3u5pky2545hqke7vz7fc3inpso5gkune 139908

[RubenKelevra]

Hey @thibaultmol,

I'm already planning to include the Manjaro packages into the cluster as well. Bandwidth and space requirements are minimal since the Manjaro packages are basically just older arch ones.

There was some issues how I did the import previously which wasn't compatible with Manjaro, so I rewrote my script around 2 month ago.

But since never one showed up even interested in accessing them, I never started to import.

Should just take some days to get everything up and running in my cluster, so you could just join.

Would that be a good solution for you? 🤔

[thibaultmol]

Sounds excellent! Using IPFS for package distribution makes so much sense and I've been excited about IPFS ever since I learned about it. Can't wait!

On Thu, 10 Dec 2020 at 19:15, @RubenKelevra notifications@github.com wrote:

Hey @thibaultmol https://github.com/thibaultmol,

I'm already planning to include the Manjaro packages into the cluster as well. Bandwidth and space requirements are minimal since the Manjaro packages are basically just older arch ones.

There was some issues how I did the import previously which wasn't compatible with Manjaro, so I rewrote my script around 2 month ago.

But since never one showed up even interested in accessing them, I never started to import.

Should just take some days to get everything up and running in my cluster, so you could just join.

Would that be a good solution for you? 🤔

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/RubenKelevra/pacman.store/issues/37#issuecomment-742701089, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHWUMTD52KOQVK3RHGWTSDSUEF4PANCNFSM4URBNBPA .

[RubenKelevra]

Alright. I start working on it tomorrow and keep you posted here. :)

[RubenKelevra]

@thibaultmol

Sorry for the huge delay, there were some higher priority tasks - for example, performance issues on the server. I couldn't add more load to it but had to change the setup.

When there's still interest in this, I can work on the small changes necessary to process a Manjaro mirror and upgrade the server for more storage to hold it too. Else it would be a bit tight.

[RubenKelevra]

upgrade the server for more storage

Done

[RubenKelevra]

I've added manjaro and endeavours to the mirror. Both yet untested.

I need to tidy up some stuff, then they will automatically be updated as well. Currently it's a manual call :)

Config (work in progress and probably still errors in it):

https://github.com/RubenKelevra/pacman.store/tree/add-further-manjaro/endeavouros-infos

I noticed also some performance issues in the last days. Will investigate that. So don't be disappointed with the current performance :)

[RubenKelevra]

sync is running now.

Please create a new bugreport if you find any issues. :)