Open toreanderson opened 4 years ago
yeah I also noticed this issue , why other mirror is not the main mirror ? Thank you
The mirror is already reported. I detected various mirrors down recently (I reported it; and some hours they solved); I am not sure the cause; maybe overload in this pandemic in some mirrors.
@sergiomb2 Issues in mirrorlist + unnecessary space with debug files... Our second mirror reduced quota... (Ho yes, we have actually big size) :smiley_cat:
The UnitedRPM mirrors have been very slow since way before the pandemic struck, for what it is worth. Even with the vorboss one disabled, the UnitedRPMs repo is still is way slower than the much larger fedora repo. To me it seems like the SourceForge mirror network is just very slow overall.
[:~] $ time sudo dnf -v --refresh --disablerepo='*' --enablerepo=unitedrpms update
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade
DNF version: 4.2.21
cachedir: /var/cache/dnf
User-Agent: constructed: 'libdnf (Fedora 32; generic; Linux.x86_64)'
unitedrpms 32 - x86_64 104 B/s | 870 B 00:08
reviving: 'unitedrpms' can be revived - repomd matches.
unitedrpms: using metadata from fr. 15. mai 2020 kl. 10.11 +0200.
Completion plugin: Generating completion cache...
--> Starting dependency resolution
--> Finished dependency resolution
Dependencies resolved.
Nothing to do.
Complete!
real 0m9,108s
user 0m0,790s
sys 0m0,134s
[:~] $ time sudo dnf -d 10 --refresh --disablerepo='*' --enablerepo=fedora update
timer: config: 2 ms
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade
DNF version: 4.2.21
Command: dnf -d 10 --refresh --disablerepo=* --enablerepo=fedora update
Installroot: /
Releasever: 32
cachedir: /var/cache/dnf
Base command: update
Extra commands: ['-d', '10', '--refresh', '--disablerepo=*', '--enablerepo=fedora', 'update']
User-Agent: constructed: 'libdnf (Fedora 32; generic; Linux.x86_64)'
countme: no event for fedora: window already counted
Fedora 32 - x86_64 20 kB/s | 24 kB 00:01
reviving: 'fedora' can be revived - metalink checksums match.
fedora: using metadata from to. 23. april 2020 kl. 00.22 +0200.
timer: sack setup: 2265 ms
Completion plugin: Generating completion cache...
--> Starting dependency resolution
--> Finished dependency resolution
timer: depsolve: 509 ms
Dependencies resolved.
Nothing to do.
Complete!
Cleaning up.
real 0m3,460s
user 0m2,010s
sys 0m0,245s
(Ho yes, we have actually big size)
What we are talking about ? 100 Gigas ? , so just mirror non-debug rpms on https://osdn.net/projects/unitedrpms/ ... I guess , It is possible , is it possible .
Anyway mirrorlist issues was fixed Anyway 2, metalink with mirror manager are now in use in Fedora and RPMFusion which is the preferred choice, because works well
Thanks
@sergiomb2 aprox... we don't need debug files (checking the traffic of each debug). Next step, debug of the debug?... :smile: Yes is possible avoid debug (some hours needs).
About metalink; in Fedora and other third-party; is irrelevant; the people choice the useful for him/her... and UnitedRPMs has great and big software.
@toreanderson If anybody want speed mirrors, collaborate with it is a great idea (We need mirrors)... the mirrors in SF uses automatic geographic election; and here we only can report it. What do you expect from a self-sustaining and collaborative infrastructure? :smile:
@sergiomb2 aprox... we don't need debug files (checking the traffic of each debug). Next step, debug of the debug?... smile Yes is possible avoid debug (some hours needs).
Debug files are separated because we save a lot of bandwidth , disk and even performance etc and for common use is rare that someone install it and we can leave without it. if we don't want debug app crashes .
we just need it to debug a crash / segfault , to find the symbols and lines of code that make the crash of the computer .
I sometime install it , usually on Beta version under Virtualbox, but after debug it , I remove it again to save disk space and bandwith , yeah I bearably update debug packages after test it I remove it , it always gigas and gigas of downloads .
https://github.com/UnitedRPMs/unitedrpms/blob/master/unitedrpms.repo#L4 where we can use
baseurl=https://osdn.net/projects/unitedrpms/storage/$releasever/$basearch/
hum and why osdn.net is not in first on https://raw.githubusercontent.com/UnitedRPMs/unitedrpms/master/mirrorlist_x86_64.txt ? I use mirrorlist in my dnf configuration btw
and we may remove debug files from osdn.net if we remove it also from here https://github.com/UnitedRPMs/unitedrpms/blob/master/mirrorlist_debug_30.txt . and maybe do the same to the sources rpms ?
What do you think ?
What do you expect from a self-sustaining and collaborative infrastructure? smile
To be clear, I do not have any demands or hard expectations per se. I am fully aware that I get what I pay for when using UnitedRPMs. So the reason for submitting this issue is just to make you aware of it, in case you weren't already. If the conclusion is that «yeah, SourceForge is slow, but it is nevertheless the best free alternative, so we'll keep using it», then that is fair enough as far as I am concerned.
@toreanderson thank you for these words
@toreanderson I am testing some changes; we need secure the changes. Thanks.
@sergiomb2 I need secure the limits and if you orbserve, osdn is incomplete (repodata) and others. But is possible enable again the mirrorlist; I remember is programmed a sync in repositories, cross fingers with some lines changed in the sync task.
@toreanderson Done. Can you test now the speed?
It seems much faster now! The sudo dnf -v --refresh --disablerepo='*' --enablerepo=unitedrpms update
commands seems to complete in around 4 seconds now, which is a major speedup compared to how it used to be.
However, it is still (slightly) slower than the much larger fedora
repo. While watching the below screen recording (play locally with asciinema play https://asciinema.org/a/irlpk0mlsiA2kvulym5vY81M4
), you can see how the progress bar starts out stalled with --- B/s
transfer speed). Indeed, it seems like the vast majority of the dnf
execution time is spent in that state - once it starts moving, it finishes almost instantaneously.
By comparison, when updating the fedora
repo, things start moving right away.
Unfortunately I have not found out how to make dnf
spit out sufficient amounts of debugging info to figure out what exactly it is waiting for when it is stalled. Any tips here would be appreciated.
@toreanderson Hi, thanks. Is only conjectures about the delay; the repository/repodata is signed (for security). Read about it and about how to works repo_gpgcheck here:
https://linux.die.net/man/5/yum.conf#repo_gpgcheck
The SourceForge mirrors seem to be extremely slow or even totally unreachable. This makes UnitedRPMs slow down essentially all
dnf
operations.It's been like this for a very long time. Years. I've held back on submitting an issue since it is not really an UnitedRPMs bug per se, but it makes having the UnitedRPMs repo enabled by default very painful. Perhaps it would be possible to find another mirror system.
For the record, I have a fast Internet connection (500Mbps FTTH), and all other DNF repos (both the upstream Fedora ones as well as multiple other third-party ones). It is only the UnitedRPMs repos that are slow.
A few examples from today:
curl simply getting a connect timeout
(I got this URL from a failing
dnf
command, for what it is worth.)same command as above retried, but this time it eventually connected, but hit some other timeout
dnf giving up on downloads because of transfer speed dropping below 1Kb/s
dnf update with only unitedrpms repo enabled:
timer: sack setup: 17664 ms
compare the above with only the (much larger!) fedora repo enabled:
timer: sack setup: 1900 ms