zstd 支持

[petronny]

话说之前提到的默认用zstd做pacman默认压缩的事情咋样了啊。。。

什么时候换zstd啊，我们可以提前跟进么？

[lilydjwg]

有 pacman 使用 xz 和 zstd 的对比数据么？ 我不是很想换，因为 xz 被支持得更好。

[petronny]

有的。当时的邮件列表里有这个

https://lists.archlinux.org/pipermail/arch-dev-public/2019-March/029542.html

是zstd多核和xz的单核对比。

如果是zstd和xz的多核顶级参数对比的话，我做过一次。我记得压缩上空间差不太多，zstd比xz要慢一些(cuda好像是xz 2:00 zstd 2:30这种吧)。解压上可以参考那个列表里的，zstd比xz快太多。

xz多核主要是被喷有不可复现的问题。主要是指xz单核和多核结果不一致（但是单核和多核独立都是可以复现的）。

[lilydjwg]

好耶～ zstd -c -T0 --ultra -20 - 这个大概是最合适的了。

接下来是测试和实现。archrepo2 和 lilac 都要做相应的修改。

[lilydjwg]

makepkg 看起来不支持呢……

[petronny]

我打了个unzip没问题啊。。。

  -> 正在压缩软件包...
==> 警告： '' 是无效的压缩包扩展名。
==> 正在离开 fakeroot 环境。
==> 完成创建：unzip 6.0-13 (2019年06月20日 星期四 13时33分36秒)

除了会出这么个警告。

我在原来的/etc/makepkg.conf下加了

PKGEXT='.pkg.tar.zst'
COMPRESSZST=(zstd -c -T0 --ultra -21 -)

出来的包我也装了一下，看起来没问题

[farseerfc]

現在的 makepkg 改了 PKGEXT='.pkg.tar.zst' 之後打出來的文件貌似是沒壓縮的 tar 。 需要等這個 commit 包含在新一版的 pacman https://git.archlinux.org/pacman.git/commit/?id=bae74c8e9e69b4f4e1a235eef21b9b27fb14aff0 以及 repo-add 的支持在 https://git.archlinux.org/pacman.git/commit/?id=7afe51171fe063bf3031cc68fc8c7ac914a01de2

[petronny]

現在的 makepkg 改了 PKGEXT='.pkg.tar.zst' 之後打出來的文件貌似是沒壓縮的 tar 。

原来如此

[petronny]

所以我们是不是装pacman-git就可以搞这个了？

从用户使用的角度有需要等待的么？

[petronny]

我试着用pacman-git打包了一个unzip-6.0-13-x86_64.pkg.tar.zst然后用pacman装，貌似没什么问题？

[yan12125]

pyalpm, even pyalpm-git, is not compatible with pacman-git yet, and apparently pyalpm is used in many scripts in archlinuxcn.

EDIT: discussions about pacman-git and pyalpm-git can be found on AUR https://aur.archlinux.org/packages/pyalpm-git

EDIT 2: namcap relies on pyalpm, too. Using pacman-git in chroots may result in flooded lilac logs.

[petronny]

这样。。。

[yan12125]

Update: Arch developers plan to switch official packages to zstd in the end of this month. See https://lists.archlinux.org/pipermail/arch-dev-public/2019-December/029752.html for details.

I guess the switch will happen in devtools (/usr/share/devtools/makepkg-x86_64.conf). I propose to do the same switch in devtools-archlinuxcn then.

[yan12125]

Official devtools has been updated to use zstd as the default compression format for packages. Also, archlinuxcn/emacs-git has been packaged with zstd for more than a month (since [1]), so I'm confident that zstd will work well for [archlinuxcn]. How about doing the switch now?

[1] https://aur.archlinux.org/cgit/aur.git/commit/PKGBUILD?h=emacs-git&id=52b9930b3277ce67627b37b7f8e605e54293c625

[lilydjwg]

What needs to be done for this? Just upgrade devtools?

[yan12125]

devtools-archlinuxcn needs to be rebased against the latest devtools first.

/cc @farseerfc

[yan12125]

What needs to be done for this? Just upgrade devtools?

Searching 'pkg.tar.xz' in archlinuxcn repos, I found another place to update: https://github.com/archlinuxcn/ansible-playbooks/blob/master/roles/repo/files/nginx-repo.conf - *.pkg.tar.zst should have longer expiration time, too?

[lilydjwg]

nginx config has been updated.

[petronny]

Official devtools has been updated to use zstd as the default compression format for packages.

现在还在 [testing]

[yan12125]

Official devtools moved to [extra] [1], devtools-archlinuxcn updated [2], the build server upgraded to the latest devtools-archlinuxcn, and in the following automatic build starting at 2020/1/7 17:22:53 (UTC+8), no packages failed due to the new zstd compression format. I believe everything has been done :tada:

[1] https://git.archlinux.org/svntogit/packages.git/commit/?h=packages/devtools&id=88c9964e88ac5ba7c1e5dc5a9c167fb640bc0df0 [2] https://github.com/archlinuxcn/repo/commit/3e8b62651d8bc8cba93ac1c269f1b5adb6e97e65