api mode

[udeved]

@fhdk

Can you make it so that in api mode setbranch auto sync can be deactivated so it only set branch in conf and mirrorlist? Perhaps additional arg -N?

[fhdk]

4.2.0-dev man pacman-mirrors

pacman-mirrors -h
---snip
MISC:
---snip
  -n, --no-mirrorlist   Use to skip generation of mirrorlist
  -y, --sync            Syncronize pacman databases

4.1.4

pacman-mirrors -h
---snip
  -u, --update          pacman -Syy
  -n, --no-mirrorlist   no mirrorlist

[fhdk]

@philmmanjaro Have you had the time to check on pacman-mirrors-dev? I think it is ready for release. The lastest commits is only finetuning of the new man page

[udeved]

So -y also works in api mode? My last test to use it in chroot-run code produced a sync by default.

[fhdk]

@philmmanjaro I will check on what udeved has in mind and if possible adopt it into 4.2.0-dev

[fhdk]

@udeved

So -y also works in api mode? My last test to use it in chroot-run code produced a sync by default.

-y and -n are mutually exclusive and not limited to api-mode. The reasoning behind is that if you don't want to generate a mirrorlist then syncing pacman databases makes no sense.

The --no-mirrorlist argument causes pacman-mirrors to exit after downloading a set of files mirrors.json and status.json

The --set-branch command does not touch the mirrorlist - only conf. To have the --set-branch reflected in the mirrorlist - the mirrorlist must be generarated with eg. -g, -f1

You can think of the api-mode as an addon - the api-commands are executed before anything else but only one makes pacman-mirrors exit and that is --get-branch.

Can you give me samples on you are combining the arguments?

And maybe a liner on what you want to achieve?

[fhdk]

@philmmanjaro I have just stumpled on a regression or maybe something I have not tested - there is a bug in 4.2.0-dev

[udeved]

@fhdk

I would want to replace these two shell functions with the more native pacman-mirrors calls in api mode. I haven't committed the code changes yet, I just tested.

https://github.com/manjaro/manjaro-tools/blob/devel/lib/util.sh#L143 https://github.com/manjaro/manjaro-tools/blob/devel/lib/util.sh#L149

https://github.com/manjaro/manjaro-tools/blob/devel/bin/chroot-run.in#L100

The mechanism is without pacman-mirrors calls, I got an empty mountpoint, the pacman-mirrors.conf gets copied to mount point, along with basic folders created to run pacman. The mirror is in build cases always statically set to netzspielplatz usually. The base group for example gets pulled in chroot, after the branch is set in mirrorlist. The "problem" is setting the branch in mirrorlist, not the variable in pacman-mirrors.conf.

If it must sync with --setbranch, then all build related calls would sync twice repo dbs, one by pacman-mirrors the other by chrooted pacman calls to pull in packages. The same mechanism is used by calamares modules.

I could change the chrooted pacman calls to use only pacman -S $pkglist instead of pacman -Sy $pkglist to work around that, but it would be very unsafe, and the sync would rely on pacman-mirrors.

Ideally, pacman-mirrors could set branch variable and adapt branch in mirrorlist without sync too.

[fhdk]

Commit https://github.com/manjaro/pacman-mirrors/commit/a655719a77a5584937a5326d93341a13ab15ea3a

I have made some changes to achieve what you describe - so a command like this

pacman-mirrors -anRS $branch -p $mnt

Explained

• -a api-mode
• -n dont generate mirrorlist
• -R replace branch in mirrorlist with $branch
• -S replace branch in config with $branch
• -p prefix files with $mnt

The api commands are prioritized

1. set prefix All commands hereafter will reflect the prefix supplied or defaults if not
2. write branch to config
3. replace branch mirror list
4. write protocols to config
5. return branch from config and exit

[fhdk]

@philmmanjaro

When you have some time to spare I would appreciate a review to spot if I missed something.

[udeved]

@fhdk

Is the commit you posted packaged yet?

[fhdk]

No it is not Current dev version is from the 18.

[udeved]

Ok, I'll rebuild the package, I need it in repos to test in chroot.

[philmmanjaro]

I've updated the package to the current changes made.

[udeved]

@fhdk

How do I stop it from generating the mirrorlist?

$ sudo pacman-mirrors -a -p /var/build/manjaro-tools/buildpkg/unstable/x86_64/root -S unstable -R 
.: Info Branch in config is changed
.: Info Downloading mirrors from repo.manjaro.org
.: Info Nutze Standard-Mirrordatei
.: Info Frage Mirrors ab - Dies kann einige Zeit in Anspruch nehmen
   0.690 Australia      : http://mirror.ventraip.net.au/Manjaro/
   0.681 Australia      : http://manjaro.uberglobalmirror.com/
   0.741 Australia      : http://manjaro.mirror.serversaustralia.com.au/
   0.113 Austria        : http://mirror.inode.at/manjaro/
   0.191 Belarus        : http://mirror.datacenter.by/pub/mirrors/manjaro/
   0.094 Belgium        : http://ftp.belnet.be/manjaro/
   0.480 Belgium        : ftp://ftp.belnet.be/mirrors/manjaro/
   0.227 Belgium        : https://manjaro.cu.be/
   0.810 Brasil         : http://linorg.usp.br/manjaro/
   0.301 Bulgaria       : https://mirrors.netix.net/manjaro/
   0.307 Bulgaria       : https://manjaro.ipacct.com/manjaro/
   0.172 Bulgaria       : http://manjaro.telecoms.bg/
   0.522 Canada         : https://ca.mirror.babylon.network/manjaro/
   0.595 Chile          : http://manjaro.dcc.uchile.cl/
   1.557 Chile          : http://doge.ing.puc.cl/Mirrors/Manjaro/
   1.489 China          : https://mirrors.ustc.edu.cn/manjaro/
   0.718 China          : http://ftp.cuhk.edu.hk/pub/Linux/manjaro/
   ..... China          : ftp://ftp.cuhk.edu.hk/pub/Linux/manjaro/
   1.026 China          : http://mirrors.tuna.tsinghua.edu.cn/manjaro/
   1.618 China          : https://mirrors.zju.edu.cn/manjaro/
   0.454 Colombia       : http://mirror.edatel.net.co/manjaro/
   0.923 Costa_Rica     : https://mirrors.ucr.ac.cr/manjaro/
   0.203 Czech          : https://mirror.dkm.cz/manjaro/
   0.222 Denmark        : https://mirrors.dotsrc.org/manjaro/
   0.395 Denmark        : https://www.uex.dk/repos/manjaro/
   0.919 Ecuador        : https://mirror.cedia.org.ec/manjaro/
   0.935 Ecuador        : https://mirror.uta.edu.ec/manjaro/
   0.925 Ecuador        : https://mirror.espoch.edu.ec/manjaro/
   0.094 France         : http://manjarolinux.polymorf.fr/
   0.116 France         : http://mirror.lignux.com/manjaro/
   0.131 France         : http://ftp.free.org/mirrors/repo.manjaro.org/repos/
   0.454 France         : ftp://ftp.free.org/mirrors/repo.manjaro.org/repos/
   0.210 France         : https://fr.mirror.babylon.network/manjaro/
   0.132 Germany        : http://mirror.ragenetwork.de/manjaro/
   0.199 Germany        : https://ftp.halifax.rwth-aachen.de/manjaro/
   0.659 Germany        : ftp://ftp.halifax.rwth-aachen.de/manjaro/
   0.067 Germany        : http://ftp.tu-chemnitz.de/pub/linux/manjaro/
   0.283 Germany        : ftp://ftp.tu-chemnitz.de/pub/linux/manjaro/
   0.260 Germany        : https://mirror.netzspielplatz.de/manjaro/packages/
   0.172 Germany        : https://mirror.netcologne.de/manjaro/
   0.178 Germany        : https://repo.rhindon.net/manjaro/
   0.178 Germany        : https://repo.stdout.net/manjaro/
   0.204 Germany        : https://mirror.alpix.eu/manjaro/
   0.238 Germany        : https://manjaro-mirror-a.alpix.eu/
   0.115 Germany        : http://ftp.rz.tu-bs.de/pub/mirror/manjaro.org/repos/
   0.550 Germany        : ftp://ftp.rz.tu-bs.de/pub/mirror/manjaro.org/repos/
   0.164 Germany        : https://manjaro.ieji.de/
   0.200 Hungary        : http://mirror.infotronik.hu/mirrors/pub/manjaro/
   0.543 Indonesia      : http://kambing.ui.ac.id/manjaro/
   0.514 Indonesia      : http://kartolo.sby.datautama.net.id/manjaro/
   0.264 Ireland        : https://ftp.heanet.ie/mirrors/manjaro/
   0.636 Ireland        : ftp://ftp.heanet.ie/mirrors/manjaro/
   0.340 Italy          : https://manjaro.mirror.garr.it/mirrors/manjaro/
   0.359 Italy          : https://ba.mirror.garr.it/mirrors/manjaro/
   0.345 Italy          : https://ct.mirror.garr.it/mirrors/manjaro/
   0.631 Japan          : http://ftp.tsukuba.wide.ad.jp/Linux/manjaro/
   0.655 Japan          : http://ftp.riken.jp/Linux/manjaro/
   ..... Japan          : ftp://ftp.riken.jp/Linux/manjaro/
   0.193 Netherlands    : https://ftp.nluug.nl/pub/os/Linux/distr/manjaro/
   0.421 Netherlands    : ftp://ftp.nluug.nl/pub/os/Linux/distr/manjaro/
   0.099 Netherlands    : http://ftp.snt.utwente.nl/pub/linux/manjaro/
   0.459 Netherlands    : ftp://ftp.snt.utwente.nl/pub/linux/manjaro/
   0.205 Netherlands    : https://nl.mirror.babylon.network/manjaro/
   0.187 Netherlands    : https://manjaro.mirror.wearetriple.com/
   0.710 Philippines    : http://mirror.rise.ph/manjaro/
   ..... Poland         : http://mirror.chmuri.net/manjaro/
.: Fehler timed out 'http://mirror.chmuri.net/manjaro/'

   0.221 Portugal       : http://ftp.dei.uc.pt/pub/linux/manjaro/
   0.867 Portugal       : ftp://ftp.dei.uc.pt/pub/linux/manjaro/
   ..... Portugal       : http://manjaro.barata.pt/
.: Fehler [Errno 111] Connection refused 'http://manjaro.barata.pt/'

   0.177 Romania        : http://mirrors.serverhost.ro/manjaro/packages/
   0.150 Romania        : http://ftp.lug.ro/manjaro/
   0.667 Romania        : ftp://ftp.lug.ro/manjaro/
   0.260 Russia         : https://mirror.yandex.ru/mirrors/manjaro/
   1.229 Singapore      : https://download.nus.edu.sg/mirror/manjaro/
   0.461 South_Africa   : http://mirror.is.co.za/mirrors/manjaro.org/
   0.219 Spain          : http://ftp.caliu.cat/manjaro/
   0.235 Sweden         : https://ftp.lysator.liu.se/pub/manjaro/
   1.314 Sweden         : ftp://ftp.lysator.liu.se/pub/manjaro/
   0.260 Sweden         : https://mirror.zetup.net/manjaro/
   0.707 Taiwan         : http://free.nchc.org.tw/manjaro/
   0.188 Turkey         : http://ftp.linux.org.tr/manjaro/
   0.828 Turkey         : ftp://ftp.linux.org.tr/manjaro/
   0.123 United_Kingdom : http://repo.manjaro.org.uk/
   0.220 United_Kingdom : https://www.mirrorservice.org/sites/repo.manjaro.org/repos/
   0.148 United_Kingdom : http://mirror.catn.com/pub/manjaro/
   0.133 United_Kingdom : http://manjaro.mirrors.uk2.net/
   0.309 United_States  : http://mirror.dacentec.com/manjaro/
   0.312 United_States  : http://mirror.jmu.edu/manjaro/
   0.311 United_States  : http://mirror.solarvps.com/manjaro/
   0.972 United_States  : http://mirror.nexcess.net/manjaro/
   0.318 United_States  : http://distro.ibiblio.org/manjaro/
   0.706 United_States  : https://mirror.clarkson.edu/manjaro/
   0.816 United_States  : https://mirrors.ocf.berkeley.edu/manjaro/
   1.506 Vietnam        : https://mirror.freedif.org/Manjaro/
.: Info Schreibe Mirrorliste
   Belgium         : http://ftp.belnet.be/manjaro/unstable
   France          : http://manjarolinux.polymorf.fr/unstable
   Austria         : http://mirror.inode.at/manjaro/unstable
   France          : http://mirror.lignux.com/manjaro/unstable
   United_Kingdom  : http://repo.manjaro.org.uk/unstable
   Germany         : http://mirror.ragenetwork.de/manjaro/unstable
   United_Kingdom  : http://manjaro.mirrors.uk2.net/unstable
   United_Kingdom  : http://mirror.catn.com/pub/manjaro/unstable
   Germany         : https://manjaro.ieji.de/unstable
   Bulgaria        : http://manjaro.telecoms.bg/unstable
   Germany         : https://mirror.netcologne.de/manjaro/unstable
   Romania         : http://mirrors.serverhost.ro/manjaro/packages/unstable
   Germany         : https://repo.rhindon.net/manjaro/unstable
   Germany         : https://repo.stdout.net/manjaro/unstable
   Netherlands     : https://manjaro.mirror.wearetriple.com/unstable
   Belarus         : http://mirror.datacenter.by/pub/mirrors/manjaro/unstable
   Hungary         : http://mirror.infotronik.hu/mirrors/pub/manjaro/unstable
   Czech           : https://mirror.dkm.cz/manjaro/unstable
   Germany         : https://mirror.alpix.eu/manjaro/unstable
   Netherlands     : https://nl.mirror.babylon.network/manjaro/unstable
   France          : https://fr.mirror.babylon.network/manjaro/unstable
   Spain           : http://ftp.caliu.cat/manjaro/unstable
   United_Kingdom  : https://www.mirrorservice.org/sites/repo.manjaro.org/repos/unstable
   Denmark         : https://mirrors.dotsrc.org/manjaro/unstable
   Belgium         : https://manjaro.cu.be/unstable
   Germany         : https://manjaro-mirror-a.alpix.eu/unstable
   Germany         : https://mirror.netzspielplatz.de/manjaro/packages/unstable
   Russia          : https://mirror.yandex.ru/mirrors/manjaro/unstable
   Sweden          : https://mirror.zetup.net/manjaro/unstable
   Germany         : http://ftp.tu-chemnitz.de/pub/linux/manjaro/unstable
   Bulgaria        : https://mirrors.netix.net/manjaro/unstable
   Bulgaria        : https://manjaro.ipacct.com/manjaro/unstable
   United_States   : http://mirror.dacentec.com/manjaro/unstable
   United_States   : http://mirror.solarvps.com/manjaro/unstable
   United_States   : http://mirror.jmu.edu/manjaro/unstable
   United_States   : http://distro.ibiblio.org/manjaro/unstable
   Italy           : https://manjaro.mirror.garr.it/mirrors/manjaro/unstable
   Italy           : https://ct.mirror.garr.it/mirrors/manjaro/unstable
   Italy           : https://ba.mirror.garr.it/mirrors/manjaro/unstable
   Denmark         : https://www.uex.dk/repos/manjaro/unstable
   Netherlands     : https://ftp.nluug.nl/pub/os/Linux/distr/manjaro/unstable
   Colombia        : http://mirror.edatel.net.co/manjaro/unstable
   France          : http://ftp.free.org/mirrors/repo.manjaro.org/repos/unstable
   Netherlands     : http://ftp.snt.utwente.nl/pub/linux/manjaro/unstable
   South_Africa    : http://mirror.is.co.za/mirrors/manjaro.org/unstable
   Belgium         : ftp://ftp.belnet.be/mirrors/manjaro/unstable
   Indonesia       : http://kartolo.sby.datautama.net.id/manjaro/unstable
   Canada          : https://ca.mirror.babylon.network/manjaro/unstable
   Indonesia       : http://kambing.ui.ac.id/manjaro/unstable
   Germany         : http://ftp.rz.tu-bs.de/pub/mirror/manjaro.org/repos/unstable
   Chile           : http://manjaro.dcc.uchile.cl/unstable
   Japan           : http://ftp.tsukuba.wide.ad.jp/Linux/manjaro/unstable
   Ireland         : https://ftp.heanet.ie/mirrors/manjaro/unstable
   Germany         : https://ftp.halifax.rwth-aachen.de/manjaro/unstable
   Romania         : http://ftp.lug.ro/manjaro/unstable
   Australia       : http://manjaro.uberglobalmirror.com/unstable
   Australia       : http://mirror.ventraip.net.au/Manjaro/unstable
   United_States   : https://mirror.clarkson.edu/manjaro/unstable
   Taiwan          : http://free.nchc.org.tw/manjaro/unstable
   Philippines     : http://mirror.rise.ph/manjaro/unstable
   Australia       : http://manjaro.mirror.serversaustralia.com.au/unstable
   Brasil          : http://linorg.usp.br/manjaro/unstable
   United_States   : https://mirrors.ocf.berkeley.edu/manjaro/unstable
   Turkey          : http://ftp.linux.org.tr/manjaro/unstable
   Portugal        : http://ftp.dei.uc.pt/pub/linux/manjaro/unstable
   Ecuador         : https://mirror.cedia.org.ec/manjaro/unstable
   Costa_Rica      : https://mirrors.ucr.ac.cr/manjaro/unstable
   Ecuador         : https://mirror.espoch.edu.ec/manjaro/unstable
   Ecuador         : https://mirror.uta.edu.ec/manjaro/unstable
   United_States   : http://mirror.nexcess.net/manjaro/unstable
   China           : http://mirrors.tuna.tsinghua.edu.cn/manjaro/unstable
   Singapore       : https://download.nus.edu.sg/mirror/manjaro/unstable
   Sweden          : https://ftp.lysator.liu.se/pub/manjaro/unstable
   China           : https://mirrors.ustc.edu.cn/manjaro/unstable
   Vietnam         : https://mirror.freedif.org/Manjaro/unstable
   Chile           : http://doge.ing.puc.cl/Mirrors/Manjaro/unstable
   China           : https://mirrors.zju.edu.cn/manjaro/unstable
   Japan           : http://ftp.riken.jp/Linux/manjaro/unstable
   China           : http://ftp.cuhk.edu.hk/pub/Linux/manjaro/unstable
.: Info Mirror list generated and saved to: /var/build/manjaro-tools/buildpkg/unstable/x86_64/root/etc/pacman.d/mirrorlist

Would be totally sufficient if the code stops here optionally:

.: Info Branch in config is changed

Perhaps, if a mirrorlist exists, eg static list with one build mirror, then execute Rebranch

[udeved]

Or, if I was able to supply additionally a specific mirror, I could use fasttrack -f 1 and I would not even have to write it to the empty folder for the mkchroot.

[fhdk]

If you actually looked at the sample a couple of post up :smile_cat:
You forgot -n argument which will exit with no mirrorlist

sudo pacman-mirrors -a -n -R  -p /var/build/manjaro-tools/buildpkg/unstable/x86_64/root -S unstable

[udeved]

$ sudo pacman-mirrors -a -p /var/build/manjaro-tools/buildpkg/unstable/x86_64/root -S unstable -R -n
.: Info Branch in config is changed
.: Info Downloading mirrors from repo.manjaro.org
.: Info The mirrors has not changed

$ sudo pacman-mirrors -a -p /var/build/manjaro-tools/buildpkg/unstable/x86_64/root -S stable -R -n
.: Info Branch in config is changed
.: Info Downloading mirrors from repo.manjaro.org
.: Info The mirrors has not changed

It rebranches, yes?

[fhdk]

Among other things the pacman-mirrors installation creates the path

/etc/pacman.d/

as a part of the installation process.

snip from setup.py in the pacman-mirrors project

   data_files=[('/etc', ['conf/pacman-mirrors.conf']),
                ('/etc/pacman.d', []),
                ('share/man/man8', ['man/pacman-mirrors.8.gz']),
                ('share/pacman-mirrors', ['share/mirrors.json']),

[fhdk]

At least it should - Your output is wrong - you're missing the info

Branch in mirrorlist is changed

[fhdk]

I need to revisit the code hang on

[udeved]

Thing is, I was playing with the idea to only use pacman-mirrors.

Currently, in order to set up a chroot, it will copy a default pacman-mirrors.conf from manjaro-tools data dir to an emtpty chroot destination, and sed the branch to the desired branch. Once the pacman-mirrors.conf is present in empty chroot, a mirrorlist is written, with only one build mirror and its corresponding branch. After this was done, packages can be installed in the empty chroot with pacman.

[fhdk]

I am deeply sorry - somehow the actual rebranch-call has slipped without me noticing. I have fixed it with commit https://github.com/manjaro/pacman-mirrors/commit/47d979df0b9015752a34e0c2d8054b0a88c3cba2

[udeved]

Would it be possible in api mode to have an additional mirror arg? Ie I can supply pacman-mirrors with a specific mirror and -nR still works?

[fhdk]

It is - the cogs are turning - but I am done for today - it was phone constant telling me mails popped in an I thought it was you :racehorse:

I will work out a -M --mirror arg tomorrow.

[udeved]

Sounds nice, greatly appreciated, because it provides indeed an api that solves a task you have to code in every little project that involves creating chroots.

[udeved]

Put it on halt @fhdk for some days , I am currently making a decision if I want to further contribute to manjaro.

[fhdk]

That was an overnight change :disappointed:

So will others benefit from the changes made

---

Configuration flow of pacman-mirrors

At launch an internal default configuration is setup, file configuration is applied and then the commandline is parsed and applied.

API arguments

These arguments modifies key elements of pacman-mirrors configuration according to the packagers needs.

The actions performed by the API are in strict order and performed before any other actions.

1. If --prefix

   • prefix internal file configuration with $PREFIX
   • continue

2. If --set-branch

   • apply internal branch configuration with $BRANCH
   • continue

3. If --url $URL must end with /

   • apply internal configuration to a mirrorlist with $URL
   • continue

4. If --re-branch

   • replace branch in mirrorlist with $BRANCH
   • continue

5. If --get-branch

   • sys.exit(config.branch)

When done pacman-mirrors checks the internet connection and download the latest datafiles for creating the mirrorlist. At this point it is possible to interupt further processing.

If the -n / no-mirrorlist argument is present pacman-mirrors will now exit.

[philmmanjaro]

Current code changes are released as 4.2.0dev.20170601-1

[fhdk]

@philmmanjaro Sorry - and thanks - just that I was disturbed and had not committed the changes

[udeved]

@fhdk

Its definitely useful for anyone who want to create a chroot, its not lost effort. Anyway, if my contributions are not appreciated, and only tolerated, kind of evil step child, I think I need to reconsider if I want to work on manjaro further.

[fhdk]

Anyway, if my contributions are not appreciated

Don't sell yourself short - I think you're more appreciated than you think - mankind simply is not in a habit of expressing appreciation - generally speaking of course.

As an example - your input has been highly valued in the evolution of pacman-mirrors.

[udeved]

This is better discussed non publicly, but I don't have any problem with you. You did a fine job in my view with pacman-mirrors. ;)

[fhdk]

pacman-mirrors does as expected

>>> sudo pacman-mirrors -a -R -Sunstable -U https://www.uex.dk/repos/manjaro/ -n
[sudo] adgangskode for fh: 
.: Info Branch in config is changed
.: Info Writing mirror list
.: Info Mirror list generated and saved to: /etc/pacman.d/mirrorlist
.: Info Branch in mirror list is changed
.: Info Downloading mirrors from repo.manjaro.org
.: Info The mirrors has not changed

Kind of misleading .: Info The mirrors has not changed since in fact it has

With this addition you can skip the -R and have the same result

[fhdk]

@philmmanjaro I would like to request a package on latest commit

[philmmanjaro]

Latest code released as 4.2.0dev.20170601-2.

[udeved]

@fhdk

Little problem... but almost there with relying completely on pacman-mirrors. Very nice so far.

$ buildpkg -p sysvinit -xwdo
 --> Loading compiler settings: x86_64
==> Removing chroot copy [/var/build/manjaro-tools/buildpkg/unstable/x86_64/root]...done
 --> pm_args: -a -R -p /var/build/manjaro-tools/buildpkg/unstable/x86_64/root -S unstable -n -U http://mirror.netzspielplatz.de/manjaro/packages/
==> Creating install root at /var/build/manjaro-tools/buildpkg/unstable/x86_64/root
 --> Copyig pacman-mirrors.conf: /usr/share/manjaro-tools/pacman-mirrors.conf /var/build/manjaro-tools/buildpkg/unstable/x86_64/root/etc/pacman-mirrors.conf
 --> Configuring pacman-mirrors: unstable
.: Info Branch in config is changed
.: Info Writing mirror list
   pkgbuild        : http://mirror.netzspielplatz.de/manjaro/packages/unstable
.: Info Mirror list generated and saved to: /var/build/manjaro-tools/buildpkg/unstable/x86_64/root/etc/pacman.d/mirrorlist
.: Info Branch in mirror list is changed
.: Info Downloading mirrors from repo.manjaro.org
Traceback (most recent call last):
  File "/usr/bin/pacman-mirrors", line 55, in <module>
    pm.run()
  File "/usr/lib/python3.6/site-packages/pacman_mirrors/pacman_mirrors.py", line 561, in run
    httpfn.update_mirrors(self.config, quiet=self.quiet)
  File "/usr/lib/python3.6/site-packages/pacman_mirrors/httpfn.py", line 200, in update_mirrors
    result = download_mirrors(config)
  File "/usr/lib/python3.6/site-packages/pacman_mirrors/httpfn.py", line 58, in download_mirrors
    if not filecmp.cmp(tempfile, config["mirror_file"]):
  File "/usr/lib/python3.6/filecmp.py", line 52, in cmp
    s2 = _sig(os.stat(f2))
FileNotFoundError: [Errno 2] No such file or directory: '/var/build/manjaro-tools/buildpkg/unstable/x86_64/root/usr/share/pacman-mirrors/mirrors.json'
  -> Installing packages to /var/build/manjaro-tools/buildpkg/unstable/x86_64/root

Could the json also be downloaded or skipped? I start with no packages installed for chroot, ie no pacman-mirrors package installed yet. I think skipping it would be best, because if pacman installs afterwards pacman-mirrors package, it will throw a filesystem conflict, since mirrors.json already exists in chroot. The -U option doesn't seem to sanitize path, I need to add a slash(/) to the url?

[udeved]

If I download the json and delete it in chroot after pacman-mirrors ran, it looks like this:

$ buildpkg -p sysvinit -xwdo
 --> Loading compiler settings: x86_64
==> Removing chroot copy [/var/build/manjaro-tools/buildpkg/unstable/x86_64/root]...done
 --> pm_args: -a -R -p /var/build/manjaro-tools/buildpkg/unstable/x86_64/root -S unstable -n -U http://mirror.netzspielplatz.de/manjaro/packages/
==> Creating install root at /var/build/manjaro-tools/buildpkg/unstable/x86_64/root
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  7919  100  7919    0     0  26563      0 --:--:-- --:--:-- --:--:-- 26484
 --> Configuring pacman-mirrors: unstable
.: Info Branch in config is changed
.: Info Writing mirror list
   pkgbuild        : http://mirror.netzspielplatz.de/manjaro/packages/unstable
.: Info Mirror list generated and saved to: /var/build/manjaro-tools/buildpkg/unstable/x86_64/root/etc/pacman.d/mirrorlist
.: Info Branch in mirror list is changed
.: Info Downloading mirrors from repo.manjaro.org
  -> Installing packages to /var/build/manjaro-tools/buildpkg/unstable/x86_64/root
:: Synchronizing package databases...
 core                                        140.5 KiB   937K/s 00:00 [#######################################] 100%
 extra                                      1803.2 KiB  1478K/s 00:01 [#######################################] 100%
 community                                     4.2 MiB  1449K/s 00:03 [#######################################] 100%
resolving dependencies...
looking for conflicting packages...

But, this doesn't work for buildiso.

[fhdk]

I can see it. Let me look at it tomorrow. we will find a solution.

What is notvworking for build iso?

[udeved]

Its ok for buildiso too, best way would be to skip the json thingy on -U switch

[fhdk]

I have made some changes to the code and I will describe them:

As described above the supplied branch has been applied to the internal configuration before applying the url argument.

So when using -U the -R is obsolete since -U will apply $BRANCH from the config and then $repo/$arch and with the current refactor pacman-mirrors will exit when -U is used so the -n argument is obsolete too.

Your args will then be

pm_args: -a -p /var/build/manjaro-tools/buildpkg/unstable/x86_64/root -S unstable -U http://mirror.netzspielplatz.de/manjaro/packages/

The changes as been commited https://github.com/manjaro/pacman-mirrors/commit/08b1b6bd58878c251cabc80966cecfaf8e318c9b but I need some mentoring using boxit to upload the changes.

[udeved]

A pretty cool addition would be similar to pacman a -C $config option, where pacman-mirrors.conf can be located any place to avoid copying the file to empty root.

[fhdk]

I don't understand - how would that be of use?

At the point where you execute pacman-mirrors outside chroot - supplying --api and a prefix, a branch and a mirror - pacman-mirrors don't care about pacman-mirrors.conf

• creates $PREFIX/etc/pacman.d - if it does not exist https://github.com/manjaro/pacman-mirrors/commit/0fce51cd8cbefa84d3026d71720e12f119d1686a
• creates a mirror list with Server = $URL/$BRANCH/$repo/$arch
• exit from execution - the conf is not needed nor read

All you have to do next is exec pacman in your chroot. You don't have to preload anything onto the chroot just

pacman-mirrors -a -p $prefix -S $branch -U $mirror

which - on an empty chroot - will result in

/etc/pacman.d/mirrorlist

with content

Server = $mirror/$branch/$repo/$arch

ping @udeved

[fhdk]

I found that the branch thingy is not working correct. The -S does not use the correct branch - I am looking into it right now

[fhdk]

Correction - it does - I have made another minor adjustment. When using -S in combination with -U, branch is only used to create the mirrorlist and is not written to config - thus eliminating the need for chroot/etc/pacman-mirrors.conf.

[fhdk]

And I have also uploaded an updated pacman-mirrors-dev package :tada:

pacman-mirrors -a -p /what/ever/you/like -S branch -U proto://mirror

creates a file

/what/ever/you/like/etc/pacman.d/mirrorlist

with content

Server = proto://mirror/branch/repo/arch

[udeved]

@fhdk

The key is, pacman is not executed in the chroot, because it don't exist yet there. Pacman is executed on the host system, and it has --root set, and --config, ie pacman.conf.

Being able to specify a pacman-mirrors.conf would make copying pacman-mirrors.conf to empty chroot obsolete. Afaik, pacman-mirrors.conf is only eeded to set the branch in chroot persistently, and pacman-mirrors package would not install default pacman-mirrors.conf.

man pacman --config <file> Specify an alternate configuration file.

I'll test the changes. Another issue to solve is that upon installing pacman-mirrors, the package generates a mirrorlist, which is unwanted in chroot, as it overwrite the set build mirror.

[fhdk]

Sorry - I must be retarded - I still don't understand.

where pacman-mirrors.conf can be located any place to avoid copying the file to empty root.

Why the need for telling pacman-mirrors where to find its conf file when your need is to create a mirrorlist with a specific branch and a specfic mirror in an empty chroot.

That is exactly what pm can do now.

In this specific api case pacman-mirrors does not care about the pacman-mirrors.conf and I can't see what info from pacman-mirrors.conf could possibly be of any value in that regard.

When it comes to - when pacman-mirrors has been installed - you can still use a 'remote' pacman-mirrors to modify the installed defaults by skipping -U argument and adding --no-mirrorlist

[udeved]

It didn't work yesterday, I still needed pacman-mirroors.conf in chroot to write mirrorlist with the buildmirror but I haven't tried latest changes yet. Looks like it may run now without a pm-conf present.

The new feature just make it new approach to creating the chroot, its kind of fine tuning now, but I like it, since this will come handy in calamares chrootcfg module too.

[fhdk]

I have made some changes since yesterday - fine tuning pacman-mirrors

[fhdk]

try using the latest dev - and check the man page or the docs/index.md page in the repo