Closed kouros17 closed 6 years ago
@kouros17: which pacman-mirrors do you use? @fhdk: did we missed some?
I use pacman-mirrors 4.5b1-2.
I will look at IT tomorrow.
It is without doubt a question of a too limited custom-mirror pool.
I do not know how pamac does its thing.
Have you tried to have a look at pamac output window or done the command in terminal?
I am going to sleep now.
~ >>> pacman-mirrors -v
Version 4.5b1
Pamac v 6.2.2-1
As said before I don't know the inner workings of pamac but pacman - I know inside out.
And with the change in b1 - the OnlyCountry =
has been deprecated and pacman-mirrors verifies the existence of a /var/lib/pacman-mirrors/custom-mirrors.json
and IF valid will generate your mirrorlist from that pool ELSE using default /var/lib/pacman-mirrors/status.json
or /usr/share/pacman-mirrors/mirrors.json.
If your mirror pool does not contain any mirrors which is up-to-date no changes will be made to the mirrorlist. That will be visible in the details window of pamac.
~ >>> sudo pacman-mirrors -c Denmark
[sudo] password for fh:
.: INFO Downloading mirrors from repo.manjaro.org
.: INFO User generated mirror list
--------------------------
.: INFO Custom mirror file saved: /var/lib/pacman-mirrors/custom-mirrors.json
.: INFO Using default mirror file
.: INFO Querying mirrors - This may take some time
0.076 Denmark : https://mirrors.dotsrc.org/manjaro/
0.072 Denmark : https://www.uex.dk/public/manjaro/
.: INFO Writing mirror list
Denmark : https://www.uex.dk/public/manjaro/unstable
Denmark : https://mirrors.dotsrc.org/manjaro/unstable
.: INFO Mirror list generated and saved to: /etc/pacman.d/mirrorlist
This command generated a custom-mirrors.json which made no change to the pamac country listing. Using the refresh mirrorlist in pamac does work well too
When you have this issue - I would like you to post the contents of
/etc/pacman-mirrors.conf
/etc/pacman.d/mirrorlist
/var/lib/pacman-mirrors/custom-mirrors.json
Also - I was thinking - do pamac use the OnlyCountry
setting? That setting was removed from the pacman-mirrors.conf
delivered with the package and installed as pacman-mirrors.conf.pacnew
.
But if you have not replaced your existing config - pacman-mirrors does NOT read/write the OnlyCountry
setting only custom-mirrors.json
.
@fhdk: thx for your detailed explanation. We have to see if we have here an error.
I have tried to force the issue but seems I can't - I remember though I have seen it during development. So in theory it could exist in pre b1.
Alsoe checked country listing - no issues
pacman-mirrors -l
pacman-mirrors --list
pacman-mirrors --country-list
I made a series of screen prints trying to force the issue
picture 1
picture 2
Seven mirrors (below picture) selected for the pool - yet only three in mirrorlist (above picture) only up-to-date mirrors are selected from the pool.
pamac refresh mirrorlist - no changes to mirrorlist - since the protocol in config was changed to https
and none of the mirrors from the pool offer that protocol.
pamac refresh database - seems done twice (maybe first mirror did not answer - I don't know)
@guinux @philmmanjaro @kouros17
My missing response I have been busy with other things and have not read the comment thread on the forum until now.
After reading those I know - a least I think so - exactly where the issue is buried.
Assumption
I assume that pamac writes the users country selection to the OnlyCountry =
config option so pacman knows which countries to prioritize.
Why it happens The problem arises because in 4.5b1 this exact option is not used by pacman. It is no problem to roll back - but based on the feedback - verbal or likes - it seemed a good solution just to check for the existence of the custom-mirrors.json.
Proper solution
I can without much work update the OnlyCountry
option when a user generates a custom mirror pool with -c only if the user has set a country or more it will be replaced with Custom
. This is how v4.4.1 works.
The concerns raised by users was that it was confusing that if they manually changed OnlyCountry
then pacman-mirrors changed their entry to Custom
and moved the countries to a custom mirror pool.
It lead to the conclusion that removing this config option completely - and creating a clear explanation of files and their locations would be optimal.
Note @guinux @philmmanjaro I resolve this issue - but I will also be absent on the forum - I have a contract to build an app based on my https://github.com/fhdk/eordre-app.pyqt application.
Yes, pamac reads/writes OnlyCountry
option.
I can adopt new pacman-mirrors and use -c
option to generate mirrorlist with one country but how can pamac have access to previous choosen country ?
I did not realize that pamac was using that entry in the config.
The users custom mirror pool created with -c
is saved in /var/lib/pacman-mirrors/custom-mirrors.json
in the exact same format as the status.json
in the same location.
From a separation of concerns POW it would be preferable to have the data files as the exchange point. EDIT: If you can accept it I think the shortcut solution here is for pacman-mirrors to write the selected countries to pamac.conf. The pro is that pacman-mirrors.conf stays simple and no-one bothers to change pamac.conf by hand.
## Comma separated list of countries
## found in `/var/lib/pacman-mirrors/custom-mirrors.json`
## Do not modify by hand as your change will be lost
#CountryList =
So I do see two options here
/etc/pamac.conf
to inform pamac of the users preferred countries.custom-mirrors.json
exist - use it else use status.json
.### Pamac configuration file
## When removing a package, also remove those dependencies
## that are not required by other packages (recurse option):
#RemoveUnrequiredDeps
## How often to check for updates, value in hours (0 to disable):
RefreshPeriod = 6
## When there are no updates available, hide the tray icon:
#NoUpdateHideIcon
## Allow Pamac to search and install packages from AUR:
EnableAUR
## When AUR support is enabled check for updates from AUR:
CheckAURUpdates
## AUR build directory:
BuildDirectory = /tmp
## Number of versions of each package to keep in the cache:
KeepNumPackages = 3
## Remove only the versions of uninstalled packages when clean cache:
#OnlyRmUninstalled
## Comma separated list of countries
## found in `/var/lib/pacman-mirrors/custom-mirrors.json`
## Do not modify by hand as your change will be lost
#CountryList =
@guinux @philmmanjaro I would very much like to close this one but which approach do we take?
@fhdk I think @guinux will find a proper way. If it is needed that pacman-mirrors should touch the pamac.conf we let you know.
@philmmanjaro @guinux
Ok - great - but I we should make sure 4.5b1 does not move to testing or stable. That is - remove it from unstable - replacing it with 4.4 - I will make an post explaining the reason to the incompatibility with pamac.
When a solution which works for pamac is in place we put the 4.5 on the table again.
I checked with repo - too late for that - I have found a minor bug in pacman-mirrors.
The bug caused the rewriting of the custom mirror file to fail when using -c
with another set of countries.
I want to keep pamac as simple as possible for user. This means that I would like to keep the current view in pamac with only one country selected.
Of course I could parse custom-mirrors.json
but this file can be created with multiple countries, and without all mirrors of a country if --interactive
option is used.
I would suggest for pacman-mirrors to:
OnlyCountry
variable in config file and change it to Custom
only if multiple countries are given or if --interactive
option is used.custom-mirrors.json
file only if it's required to change OnlyCountry
variable to Custom
.This way we can have a simple way to use pacman-mirrors for beginners with one country in OnlyCountry
and for advanced users which used the command line for more control, this will be changed to Custom
.
@philmmanjaro @guinux I get your idea but I still think it's a bad habit to write other apps config. This specific issue a really good argument for an app not writing to another app's config.
The specific reason for deprecating the OnlyCountry
was because it was confusing. As can be read in the forum reference in OP.
You don't even have to parse json with this approach. You still keep it simple for your users - just let them select the country and let pacman-mirrors
create the custom-mirrors.json
with the single chosen country.
Create an entry in pamac.conf
to hold a users Pamac selected country.
If entry is empty Worldwide
is assumed.
pamac.conf
.pacman-mirrors -c COUNTRY
pacman-mirrors -call
.You know it has changed if custom-mirrors.json
does not exist.
A user would and should not expect a change made by pacman-mirrors to be reflected in pamac.
As long as you let pacman-mirrors create the mirror pool you are good.
Another idea I can come up with is to create a similar output to -l
which could be -sc
which would output the selected countries from the configuration. You can then make your decisions based on that output.
@fhdk Your idea is near from one I had and is good as long as, like you said:
A user would and should not expect a change made by pacman-mirrors to be reflected in pamac
I can go with it if we all agree. @philmmanjaro what's your opinion ?
@guinux: I recommend the same as @fhdk. Use pamac.conf for saving the country and only trigger pacman-mirrors with given switches. This will guarantee no mixup within the apps.
Ok, I will go for it.
@guinux When pacman-mirrors launch it checks the file in /usr/share/pacman-mirrors. It does that by checksumming the file downloaded from github with the existing file to see if the existing file needs updating.
When your user has selected a country - and pacman-mirrors has done its thing - you could checksum the generated custom-mirrors.json
and save that checksum beside your country selection.
When pamac is launched you can check for existance of custom-mirrors.json - compare checksum if exist - and let the user know that you know the pacman-mirrors info has changed and you can ask questions or default or .....
Just an idea .... :sun_with_face:
Can you had a function to pacman-mirrors which just do the checksum and return if the mirrorlist needs to be updated? It can help me.
I use a python library filecmp
When your user has selected and pacman returns you could do
md5sum /var/lib/pacman-mirrors/custom-mirrors.json
9279c999cf9548fd3065fd3d2a1c627a /var/lib/pacman-mirrors/custom-mirrors.json
Store the checksum in a key in your conf. When user loads the preferences you could do the same command and compare the result to your stored value. If not equal then user has used pacman-mirrors to change the config and you can act accordingly.
@guinux
Can you had a function to pacman-mirrors which just do the checksum and return if the mirrorlist needs to be updated? It can help me.
How can I do that?
@guinux @philmmanjaro
This post draw my attention to this old issue.
I have been thinking on the country selection issue.
As of v4.7.6 you ask pacman-mirrors about version, countries and branch.
pacman-mirrors -v
pacman-mirrors -l
pacman-mirrors -aG
Pamac can easily set a country by using the --country
argument to pacman-mirrors.
pacman-mirrors -c {country[, country ...]}
Pamac can store that in its own config. But the mirror list result might not be as expected as the mirrors might not be up-to-date and pacman-mirrors only uses up-to-date mirrors.
For this to be useful in terms of Pamac functionality, pacman-mirrors must be enhanced with an argument which allows for mirrors not up-to-date.
Also I can enhance pacman-mirrors by adding an argument which returns the countries currently in the custom-mirrors.json.
Let me know what you think.
I have added some issues https://github.com/manjaro/pacman-mirrors/issues/126 https://github.com/manjaro/pacman-mirrors/issues/127 https://github.com/manjaro/pacman-mirrors/issues/128
@guinux
I have build a dev package and made it available here
The pacman-mirrors in the package has a new argument -lc/--country-config
which can be called without root.
It is similar to the -l/--list/--country-list
and upon execution it will return the countries listed in custom-mirrors.json
if it exists otherwise nothing.
Let me know if it works for you.
What pamac needs is to have access to all available countries and the custom choosen country.
@guinux @philmmanjaro https://github.com/manjaro/pacman-mirrors/releases/tag/4.9.0
@guinux
Usual country list
pacman-mirrors -l
Custom country list
pacman-mirrors -lc
Sample
~ >>> sudo pacman-mirrors -c Germany,France
[sudo] password for fh:
.: INFO Downloading mirrors from repo.manjaro.org
.: INFO User generated mirror list
--------------------------
.: INFO Custom mirror file saved: /var/lib/pacman-mirrors/custom-mirrors.json
.: INFO Using custom mirror file
.: INFO Querying mirrors - This may take some time
0.201 France : http://kibo.remi.lu/
0.118 Germany : http://mirror.ragenetwork.de/manjaro/
0.188 Germany : https://mirror.netzspielplatz.de/manjaro/packages/
0.160 Germany : https://mirror.alpix.eu/manjaro/
0.204 Germany : https://mirror.philpot.de/manjaro/
0.061 Germany : http://mirror.philpot.de/manjaro/
.: INFO Writing mirror list
Germany : http://mirror.philpot.de/manjaro/unstable/$repo/$arch
Germany : http://mirror.ragenetwork.de/manjaro/unstable/$repo/$arch
Germany : https://mirror.alpix.eu/manjaro/unstable/$repo/$arch
Germany : https://mirror.netzspielplatz.de/manjaro/packages/unstable
France : http://kibo.remi.lu/unstable/$repo/$arch
.: INFO Mirror list generated and saved to: /etc/pacman.d/mirrorlist
.: INFO To remove custom config run 'sudo pacman-mirrors -c all'
~ >>> pacman-mirrors -lc
France
Germany
Single country sample
~ >>> sudo pacman-mirrors -c Denmark
.: INFO Downloading mirrors from repo.manjaro.org
.: INFO Using custom mirror file
.: INFO Querying mirrors - This may take some time
0.103 Denmark : https://mirrors.dotsrc.org/manjaro/
0.079 Denmark : https://www.uex.dk/public/manjaro/
.: INFO User generated mirror list
--------------------------
.: INFO Custom mirror file saved: /var/lib/pacman-mirrors/custom-mirrors.json
.: INFO Writing mirror list
Denmark : https://www.uex.dk/public/manjaro/unstable/$repo/$arch
.: INFO Mirror list generated and saved to: /etc/pacman.d/mirrorlist
~ >>> pacman-mirrors -lc
Denmark
So with -l
we get the total list of countries available and with -c
you can define one. I don't think we will use the interactive one. @guinux does this functionality help pamac?
I will use:
pacman-mirrors -l
to display all available countries
pacman-mirrors -lc
to retrieve the previous choosen country
pacman-mirrors -c
with the new choosen country
pacman-mirrors -lc
to retrieve the previous choosen country
Please be aware of possible return values.
Changes adopted :) See 0bfe35e
Fixed in 6.2.5-4 :+1:
After the last changes of pacman-mirrors.conf, it's not possible to choose a country other than "worldwide" (pamac 6.2.2-1)