search

termux / termux-packages

A package build system for Termux.
https://termux.dev
Other
13.33k stars 3.07k forks source link

[Bug]: termux-tools: pkg command can select "stale" mirror #13977

Open xtkoba opened 1 year ago

xtkoba commented 1 year ago

Problem description

The following is a terminal output of pkg update command:

~ $ pkg update
Testing the available mirrors:
[*] (10) https://packages-cf.termux.dev/apt/termux-main: ok
[*] (1) https://linux.domainesia.com/applications/termux/termux-main: ok
[*] (1) https://mirror.nevacloud.com/applications/termux/termux-main: bad
[*] (1) https://mirrors.cbrx.io/apt/termux/termux-main: ok
[*] (1) https://mirror.bardia.tech/termux/termux-main: ok
[*] (1) https://mirror.albony.xyz/termux/termux-main: ok
[*] (1) https://packages.nscdn.top/termux-main: bad
[*] (1) https://mirrors.nju.edu.cn/termux/apt/termux-main: ok
[*] (1) https://mirror.nyist.edu.cn/termux/apt/termux-main: ok
[*] (1) https://mirrors.scau.edu.cn/termux/apt/termux-main: ok
[*] (1) https://mirrors.sdu.edu.cn/termux/termux-main: ok
[*] (1) https://mirrors.ustc.edu.cn/termux/apt/termux-main: ok
[*] (1) https://mirrors.cqupt.edu.cn/termux/apt/termux-main: ok
[*] (1) https://mirrors.njupt.edu.cn/termux/apt/termux-main: ok
[*] (1) https://mirror.iscas.ac.cn/termux/apt/termux-main: ok
[*] (1) https://mirrors.dgut.edu.cn/termux/apt/termux-main: bad
[*] (1) https://mirrors.hit.edu.cn/termux/apt/termux-main: ok
[*] (1) https://mirrors.pku.edu.cn/termux/termux-main/: ok
[*] (1) https://mirrors.sau.edu.cn/termux/apt/termux-main: ok
[*] (1) https://mirrors.bfsu.edu.cn/termux/apt/termux-main: ok
[*] (1) https://mirrors.tuna.tsinghua.edu.cn/termux/apt/termux-main: ok
[*] (1) https://mirrors.sustech.edu.cn/termux/apt/termux-main: ok
[*] (1) https://mirrors.aliyun.com/termux/termux-packages-24: bad
[*] (1) https://cdn.lumito.net/termux/termux-main: ok
[*] (4) https://grimler.se/termux/termux-main: ok
[*] (1) https://mirrors.sahilister.in/termux/termux-main: ok
[*] (1) https://termux.librehat.com/apt/termux-main: ok
[*] (1) https://termux.astra.in.ua/apt/termux-main: ok
[*] (1) https://mirror.mwt.me/termux/main: ok
[*] (1) https://termux.3san.dev/termux/termux-main: ok
[*] (1) https://mirror.accum.se/mirror/termux.dev/termux-main: ok
[*] (1) https://termux.cdn.lumito.net/termux-main: ok
[*] (1) https://termux.mentality.rip/termux-main: ok
[*] (1) https://mirror.termux.dev/termux-main: bad
[*] (1) https://plug-mirror.rcac.purdue.edu/termux/termux-main: ok
[*] (1) https://dl.kcubeterm.com/termux-main: ok
[*] (1) https://mirrors.utermux.dev/termux/termux-main: ok
[*] (1) https://mirror.mwt.me/termux/main: ok
[*] (1) https://mirror.vern.cc/termux/termux-main: ok
[*] (4) https://packages.termux.dev/apt/termux-main: ok
[*] (1) https://mirror.csclub.uwaterloo.ca/termux/termux-main: ok
[*] (1) https://mirror.surf/termux/termux-main/: ok
[*] (1) http://mirror.mephi.ru/termux/termux-main: ok
Picking mirror: (26) /data/data/com.termux/files/usr/etc/termux/mirrors/china/mirrors.tuna.tsinghua.edu.cn
Get:1 https://mirrors.tuna.tsinghua.edu.cn/termux/apt/termux-main stable InRelease [14.0 kB]
Get:2 https://mirrors.tuna.tsinghua.edu.cn/termux/apt/termux-main stable/main aarch64 Packages [478 kB]
Fetched 492 kB in 3s (156 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
11 packages can be upgraded. Run 'apt list --upgradable' to see them.

In the list, we see mirrors.nju.edu.cn is marked as ok:

[*] (1) https://mirrors.nju.edu.cn/termux/apt/termux-main: ok

This mirror, however, is not kept up-to-date. I noticed that when I tried to install mariadb package.

What steps will reproduce the bug?

Strictly this is hard to reproduce because pkg command seems to select the mirror randomly. But we can manually modify $PREFIX/etc/apt/sources.list and see what happens:

~ $ cat $PREFIX/etc/apt/sources.list
deb https://mirrors.nju.edu.cn/termux/apt/termux-main/ stable main
~ $ apt-get update
Hit:1 https://mirrors.nju.edu.cn/termux/apt/termux-main stable InRelease
Reading package lists... Done
~ $ apt-get install mariadb
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libedit openssl-1.1
The following NEW packages will be installed:
  libedit mariadb openssl-1.1
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 13.0 MB of archives.
After this operation, 164 MB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.

Note that the latest mariadb package in our repo does not depend on openssl-1.1.

What is the expected behavior?

Mirrors that are not kept up-to-date should not be selected by pkg command.

System information

termux-info:

Termux Variables:
TERMUX_APK_RELEASE=F_DROID
TERMUX_APP_PACKAGE_MANAGER=apt
TERMUX_APP_PID=5670
TERMUX_IS_DEBUGGABLE_BUILD=0
TERMUX_MAIN_PACKAGE_FORMAT=debian
TERMUX_VERSION=0.118.0
Packages CPU architecture:
aarch64
Subscribed repositories:
# sources.list
deb https://packages-cf.termux.org/apt/termux-main/ stable main
Updatable packages:
All packages up to date
termux-tools version:
1.34.1
Android version:
8.1.0
Kernel build information:
Linux localhost 4.9.82-perf+ #1 SMP PREEMPT Wed Feb 27 13:05:28 CST 2019 aarch64 Android
Device manufacturer:
asus
Device model:
ASUS_X01AD
2096779623 commented 1 year ago

because pkg command seems to select the mirror randomly.

https://github.com/termux/termux-packages/wiki/Mirrors#mirrors-that-are-part-of-mirror-rotation-in-pkg

This mirror, however, is not kept up-to-date

I will create an issue to report it.

licy183 commented 1 year ago

In fact, in the past, the tool auto-selected sdu.edu.cn and it is also not up to date... We've disscussed about how to solve this, see https://github.com/termux/termux-tools/issues/4. We also have a mirrorbits tracker, but it is only available when the mirror opens rsync.

xtkoba commented 1 year ago

I wonder how other distros handle this.

What I came up with is to distribute a file containing some sort of timestamp and to make pkg check it and mark the mirror as "bad" if it is older than say 1 week.

sylirre commented 1 year ago

The timestamp is a part of Release metadata file:

Origin: termux-main stable
Label: termux-main stable
Suite: stable
Codename: stable
Date: Mon, 26 Dec 2022 07:56:31 UTC
Architectures: aarch64 arm i686 x86_64
Components: main
Description: Generated by aptly
MD5Sum:
agnostic-apollo commented 1 year ago

Mentioned that at https://github.com/termux/termux-tools/issues/4#issuecomment-1358998183 with potential solution.

People here may not have gotten the notification.