Setting `CARCH` to `x86_64_v3` in `makepkg.conf` Immediately Causes Chroot Build Failures

ThisNekoGuy commented 2 years ago

Affected Version

paru v1.9.0.r13.g48fe138 - libalpm v13.0.1

Description

Setting CARCH to x86_64_v3 in makepkg.conf causes building packages in chroots to fail with the message: setarch: x86_64_v3: Unrecognized architecture However, I also run a Ryzen 3700X and I know my CPU supports it (and running makepkg normally doesn't have this issue). For reference:

neko-san@ARCH ~> /lib/ld-linux-x86-64.so.2 --help | grep supported
  x86-64-v3 (supported, searched)
  x86-64-v2 (supported, searched)
  x86_64 (AT_PLATFORM; supported, searched)
  tls (supported, searched)
  x86_64 (supported, searched)
neko-san@ARCH ~> /lib/ld-linux.so.2 --help | grep supported
  i686 (AT_PLATFORM; supported, searched)
  tls (supported, searched)
  sse2 (supported, searched)
neko-san@ARCH ~>

And to be clear, it's not because of the underscores, doing it with hyphens produces the same result.

Output

Terminal output

neko-san@ARCH ~> paru -Sa alhp-keyring alhp-mirrorlist --noconfirm
doas (neko-san@ARCH) password: 
:: Resolving dependencies...
:: Calculating conflicts...
:: Calculating inner conflicts...

Aur (2) alhp-keyring-20211125-1  alhp-mirrorlist-20211125-3

:: Proceed to review? [Y/n]: 

:: Downloading PKGBUILDs...
 PKGBUILDs up to date
fetching devel info...
[sudo] password for neko-san: 
setarch: x86_64_v3: Unrecognized architecture
error: failed to run: arch-nspawn -C /tmp/.tmptNVLOi -M /etc/makepkg.conf /mnt/extraStorage/AUR/ParuChroot/root --bind-ro /mnt/extraStorage/AUR/ParuChroot/repo/aur --bind /var/cache/pacman/pkg/ pacman -Syu --noconfirm: 
neko-san@ARCH ~ [1]>

paru.conf

#
# $PARU_CONF
# /etc/paru.conf
# ~/.config/paru/paru.conf
#
# See the paru.conf(5) manpage for options

#
# GENERAL OPTIONS
#
[options]
PgpFetch
Devel
Provides
DevelSuffixes = -git -cvs -svn -bzr -darcs -always
BottomUp
RemoveMake
SudoLoop = true
#UseAsk
#CombinedUpgrade
CleanAfter
UpgradeMenu
NewsOnUpgrade
#AurOnly

LocalRepo
Chroot = /mnt/extraStorage/AUR/ParuChroot/
#Sign
#SignDb

#
# Binary OPTIONS
#
[bin]
Sudo = doas
FileManager = dolphin
#MFlags = --skippgpcheck

makepkg.conf

#!/hint/bash
#
# /etc/makepkg.conf
#

#########################################################################
# SOURCE ACQUISITION
#########################################################################
#
#-- The download utilities that makepkg should use to acquire sources
#  Format: 'protocol::agent'
DLAGENTS=('file::/usr/bin/curl -gqC - -o %o %u'
          'ftp::/usr/bin/curl -gqfC - --ftp-pasv --retry 3 --retry-delay 3 -o %o %u'
          'http::/usr/bin/curl -gqb "" -fLC - --retry 3 --retry-delay 3 -o %o %u'
          'https::/usr/bin/curl -gqb "" -fLC - --retry 3 --retry-delay 3 -o %o %u'
          'rsync::/usr/bin/rsync --no-motd -z %u %o'
          'scp::/usr/bin/scp -C %u %o')

# Other common tools:
# /usr/bin/snarf
# /usr/bin/lftpget -c
# /usr/bin/wget
# /usr/bin/axel

#-- The package required by makepkg to download VCS sources
#  Format: 'protocol::package'
VCSCLIENTS=('bzr::bzr'
            'fossil::fossil'
            'git::git'
            'hg::mercurial'
            'svn::subversion')

#########################################################################
# ARCHITECTURE, COMPILE FLAGS
#########################################################################
#
CARCH="x86_64_v3"
CHOST="x86_64-pc-linux-gnu"

#-- Compiler and Linker Flags
export CC=clang
export CXX=clang++
export LD=lld
export CC_LD=lld
export CXX_LD=lld
export AR=llvm-ar

#CPPFLAGS=""
CFLAGS="-march=x86-64-v3 -mtune=native -O3 -pipe -fno-plt -minline-all-stringops -fexceptions -Wall \
        -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security \
        -fstack-clash-protection -fcf-protection"
CXXFLAGS="$CFLAGS -Wp,-D_GLIBCXX_ASSERTIONS"
LDFLAGS="-Wl,-O3,--sort-common,--as-needed,-z,relro,-z,now"
#LDFLAGS="-Wl -O3 -fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind"
RUSTFLAGS="-C opt-level=3 -C target-cpu=native"
#-- Make Flags: change this for DistCC/SMP systems
MAKEFLAGS="-j$(nproc)"
#-- Debugging flags
## The "-fvar-tracking-assignments" flag is for GCC
#DEBUG_CFLAGS="-g -fvar-tracking-assignments"
DEBUG_CFLAGS="-g"
#DEBUG_CXXFLAGS="-g -fvar-tracking-assignments"
DEBUG_CXXFLAGS="-g"
#DEBUG_RUSTFLAGS="-C debuginfo=2"

#########################################################################
# BUILD ENVIRONMENT
#########################################################################
#
# Makepkg defaults: BUILDENV=(!distcc !color !ccache check !sign)
#  A negated environment option will do the opposite of the comments below.
#
#-- distcc:   Use the Distributed C/C++/ObjC compiler
#-- color:    Colorize output messages
#-- ccache:   Use ccache to cache compilation
#-- check:    Run the check() function if present in the PKGBUILD
#-- sign:     Generate PGP signature file
#
BUILDENV=(!distcc color !ccache check !sign)
#
#-- If using DistCC, your MAKEFLAGS will also need modification. In addition,
#-- specify a space-delimited list of hosts running in the DistCC cluster.
#DISTCC_HOSTS=""
#
#-- Specify a directory for package building.
#BUILDDIR=/tmp/makepkg

#########################################################################
# GLOBAL PACKAGE OPTIONS
#   These are default values for the options=() settings
#########################################################################
#
# Makepkg defaults: OPTIONS=(!strip docs libtool staticlibs emptydirs !zipman !purge !debug !lto)
#  A negated option will do the opposite of the comments below.
#
#-- strip:      Strip symbols from binaries/libraries
#-- docs:       Save doc directories specified by DOC_DIRS
#-- libtool:    Leave libtool (.la) files in packages
#-- staticlibs: Leave static library (.a) files in packages
#-- emptydirs:  Leave empty directories in packages
#-- zipman:     Compress manual (man and info) pages in MAN_DIRS with gzip
#-- purge:      Remove files specified by PURGE_TARGETS
#-- debug:      Add debugging flags as specified in DEBUG_* variables
#-- lto:        Add compile flags for building with link time optimization
#
OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !debug lto)

#-- File integrity checks to use. Valid: md5, sha1, sha224, sha256, sha384, sha512, b2
INTEGRITY_CHECK=(sha256)
#-- Options to be used when stripping binaries. See `man strip' for details.
STRIP_BINARIES="--strip-all"
#-- Options to be used when stripping shared libraries. See `man strip' for details.
STRIP_SHARED="--strip-unneeded"
#-- Options to be used when stripping static libraries. See `man strip' for details.
STRIP_STATIC="--strip-debug"
#-- Manual (man and info) directories to compress (if zipman is specified)
MAN_DIRS=({usr{,/local}{,/share},opt/*}/{man,info})
#-- Doc directories to remove (if !docs is specified)
DOC_DIRS=(usr/{,local/}{,share/}{doc,gtk-doc} opt/*/{doc,gtk-doc})
#-- Files to be removed from all packages (if purge is specified)
PURGE_TARGETS=(usr/{,share}/info/dir .packlist *.pod)
#-- Directory to store source code in for debug packages
DBGSRCDIR="/usr/src/debug"

#########################################################################
# PACKAGE OUTPUT
#########################################################################
#
# Default: put built package and cached source in build directory
#
#-- Destination: specify a fixed directory where all packages will be placed
#PKGDEST=/home/packages
#-- Source cache: specify a fixed directory where source files will be cached
#SRCDEST=/home/sources
#-- Source packages: specify a fixed directory where all src packages will be placed
#SRCPKGDEST=/home/srcpackages
#-- Log files: specify a fixed directory where all log files will be placed
#LOGDEST=/home/makepkglogs
#-- Packager: name/email of the person or organization building packages
#PACKAGER="John Doe <john@doe.com>"
#-- Specify a key to use for package signing
#GPGKEY=""

#########################################################################
# COMPRESSION DEFAULTS
#########################################################################
#
COMPRESSGZ=(gzip -c -f -n)
COMPRESSBZ2=(bzip2 -c -f)
COMPRESSXZ=(xz -c -z --threads=0 -)
COMPRESSZST=(zstd -c -z -q --threads=0 -)
COMPRESSLRZ=(lrzip -q)
COMPRESSLZO=(lzop -q)
COMPRESSZ=(compress -c -f)
COMPRESSLZ4=(lz4 -q)
COMPRESSLZ=(lzip -c -f)

#########################################################################
# EXTENSION DEFAULTS
#########################################################################
#
PKGEXT='.pkg.tar.zst'
SRCEXT='.src.tar.gz'

#########################################################################
# OTHER
#########################################################################
#
#-- Command used to run pacman as root, instead of trying sudo and su
#PACMAN_AUTH=(doas)

pacman.conf

#
# /etc/pacman.conf
#
# See the pacman.conf(5) manpage for option and repository directives

#
# GENERAL OPTIONS
#
[options]
# The following paths are commented out with their default values listed.
# If you wish to use different paths, uncomment and update the paths.
#RootDir     = /
#DBPath      = /var/lib/pacman/
#CacheDir    = /var/cache/pacman/pkg/
#CacheDir     = /mnt/bigStorage/ParuChroot/repo/aur/cache/
#LogFile     = /var/log/pacman.log
#GPGDir      = /etc/pacman.d/gnupg/
#HookDir     = /etc/pacman.d/hooks/
HoldPkg     = pacman glibc
#XferCommand = /usr/bin/curl -L -C - -f -o %o %u
#XferCommand = /usr/bin/wget --passive-ftp -c -O %o %u
#CleanMethod = KeepInstalled
Architecture = i686 x86_64 x86_64_v3 znver2

# Pacman won't upgrade packages listed in IgnorePkg and members of IgnoreGroup
#IgnorePkg   =
#IgnoreGroup = modified

#NoUpgrade   =
#NoExtract   =

# Misc options
#UseSyslog
Color
#TotalDownload
#NoProgressBar
CheckSpace
#VerbosePkgLists
ParallelDownloads = 5

# By default, pacman accepts packages signed by keys that its local keyring
# trusts (see pacman-key and its man page), as well as unsigned packages.
SigLevel    = Required DatabaseOptional
LocalFileSigLevel = Optional
#RemoteFileSigLevel = Required

# NOTE: You must run `pacman-key --init` before first using pacman; the local
# keyring can then be populated with the keys of all official Arch Linux
# packagers with `pacman-key --populate archlinux`.

#
# REPOSITORIES
#   - can be defined here or included from another file
#   - pacman will search repositories in the order defined here
#   - local/custom mirrors can be added here or in separate files
#   - repositories listed first will take precedence when packages
#     have identical names, regardless of version number
#   - URLs will have $repo replaced by the name of the current repo
#   - URLs will have $arch replaced by the name of the architecture
#
# Repository entries are of the format:
#       [repo-name]
#       Server = ServerName
#       Include = IncludePath
#
# The header [repo-name] is crucial - it must be present and
# uncommented to enable the repo.
#

# The testing repositories are disabled by default. To enable, uncomment the
# repo name header and Include lines. You can add preferred servers immediately
# after the header, and they will be used before the default mirrors.

[core-x86-64-v3]
Include = /etc/pacman.d/alhp-mirrorlist

[extra-x86-64-v3]
Include = /etc/pacman.d/alhp-mirrorlist

[community-x86-64-v3]
Include = /etc/pacman.d/alhp-mirrorlist

#[testing]
#Include = /etc/pacman.d/mirrorlist

[core]
Include = /etc/pacman.d/mirrorlist

[extra]
Include = /etc/pacman.d/mirrorlist

#[community-testing]
#Include = /etc/pacman.d/mirrorlist

[community]
Include = /etc/pacman.d/mirrorlist

# If you want to run 32 bit applications on your x86_64 system,
# enable the multilib repositories as required here.

#[multilib-testing]
#Include = /etc/pacman.d/mirrorlist

[multilib]
Include = /etc/pacman.d/mirrorlist

# An example of a custom package repository.  See the pacman manpage for
# tips on creating your own repositories.
#[custom]
#SigLevel = Optional TrustAll
#Server = file:///home/custompkgs

[aur]
SigLevel = PackageOptional DatabaseOptional
Server = file:///mnt/extraStorage/AUR/ParuChroot/repo/aur/

Morganamilo commented 2 years ago

There's a flag for arch-nspawn to not set arch. I guess that would fix it. I guess setting this is for cross compiling. So there should be not too much issue setting it by default.

ThisNekoGuy commented 2 years ago

@Morganamilo I recently reported a bug about makechrootpkg about the devtools package; supposedly how it's handled is that you can leave a file in /usr/share/devtools/setarch-aliases.d/<architecture-name-file> with the contents left as "x86_64"

However, even though this will permit makechrootpkg to accept that as valid, it'll continue to produce "x86_64" labeled packages (perhaps because it may not align with the arch value set in PKGBUILDs?). Paru's implementation later expects to find the CARCH named package in the repo update but then doesn't find it because of that.

Additionally, their solution causes a similar problem for makepkg itself, because then makepkg can automatically deny builds because the targeted PKGBUILD may not have the same arch set. Which kind of makes this part of the issue on the Paru side probably related to https://github.com/Morganamilo/paru/issues/188 because it would involve overriding the PKGBUILD in some way, unless upstream decides to finally own the fact that just because it's intentionally a pain to change the CARCH doesn't mean people don't compile for their own architectures anyway and that inhibiting that just because of what Arch officially supports on paper is stupid.

ThisNekoGuy commented 2 years ago

@Morganamilo I apologize for being heated that day; I was just tired of having dispute this upstream when it shouldn't be an issue to fight about in the first place. :/

Adding that alias to setarch does let the packages build in the chroot, but since some architecture options are of course not going to be added automatically as-is, I think it would be a good idea to parse CARCH and either append or replace the value(s) for PKGBUILDs to easily solve the latter part of this problem for when it tries to update local DBs but can't because it's looking for x architecture but makechrootpkg hands us an "x86-64" named package (which might be assumed to be that because the arch variable didn't take CARCH into account).

Another option would be to just rename the produced package to x architecture so that the DB update doesn't get confused.

I'm actually trying to look into this myself but I'm new to rust.

RushingAlien commented 2 months ago

wouldn't it be -v3 instead of _v3?

skygunner commented 1 week ago

I think you shall only change -march=x86-64-v3 -mtune=native, not change CARCH.

CARCH is only for i686 and x86_64, there's no x86_64_v3 or x86_64-v3 for CARCH.

Morganamilo / paru