Raspberry Pi support?

[jlpoolen]

On a Raspberry Pi 4B running Gentoo Linux:

pizero2 /home/jlpoole # date; uname -a Wed Oct 18 04:36:37 PM PDT 2023 Linux pizero2 5.15.61-v8-

pizero2 /home/jlpoole #
pizero2 /home/jlpoole # resolve-march-native --version
resolve-march-native 2.2.0
pizero2 /home/jlpoole # resolve-march-native --debug
# gcc -S -fverbose-asm -o /tmp/tmpq6vnkz_q/march_native.s /tmp/tmpq6vnkz_q/empty.c -march=native
Flags extracted: -mabi=lp64 -mcpu=cortex-a72+crc -mlittle-endian
Flags extracted: -mabi=lp64 -march= -mbranch-protection= -mcmodel=small -mcpu=cortex-a72+crc -mfix-cortex-a53-835769 -mfix-cortex-a53-843419 -mglibc -mharden-sls= -mlittle-endian -momit-leaf-frame-pointer -moutline-atomics -moverride= -mpc-relative-literal-loads -msign-return-address=none -mstack-protector-guard-offset= -mstack-protector-guard-reg= -mstack-protector-guard=global -msve-vector-bits=scalable -mtls-dialect=desc -mtls-size=24 -mtune=
# gcc -S -fverbose-asm -o /tmp/tmpt_64q6jq/march_native.s /tmp/tmpt_64q6jq/empty.c -march=
gcc: error: missing argument to '-march='
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/resolve_march_native/__main__.py", line 50, in _inner_main
    options.gcc, options.debug).run(options)
                                ^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/resolve_march_native/engine.py", line 122, in run
    march_explicit_flag_set = self._get_march_explicit_flag_set(
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/resolve_march_native/engine.py", line 68, in _get_march_explicit_flag_set
    run(self._gcc_command, [self._get_march_explicit(arch)], self._debug)))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/resolve_march_native/runner.py", line 44, in run
    subprocess.check_output(cmd, env=env)
  File "/usr/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['gcc', '-S', '-fverbose-asm', '-o', '/tmp/tmpt_64q6jq/march_native.s', '/tmp/tmpt_64q6jq/empty.c', '-march=']' returned non-zero exit status 1.
ERROR: Command '['gcc', '-S', '-fverbose-asm', '-o', '/tmp/tmpt_64q6jq/march_native.s', '/tmp/tmpt_64q6jq/empty.c', '-march=']' returned non-zero exit status 1.
pizero2 /home/jlpoole #  eix resolve-march-native -I
[I] app-misc/resolve-march-native
     Available versions:  (~)1.0.0-r1^t (~)2.1.0^t (~)2.1.0-r1^t (~)2.2.0^t{gpkg} {test PYTHON_TARGETS="python3_10 python3_11"}
     Installed versions:  2.2.0^t{gpkg}(08:19:32 AM 10/18/2023)(-test PYTHON_TARGETS="python3_11 -python3_10")
     Homepage:            https://github.com/hartwork/resolve-march-native
     Description:         Resolve GCC flag -march=native

pizero2 /home/jlpoole # 

I saved the following: script of emerge session as resolve-march-native_20231918_0815.script.txt [resolve-march-native_20231918_0815.script.txt](https://github.com/hartwork/resolve-march-native/files/13035021/resolve-march-native_20231918_0815.sc environment file: environment.txt ript.txt) build.log.gz as build.log.gz

[hartwork]

Hi @jlpoolen, this part looks off:

[..]
# gcc -S -fverbose-asm -o /tmp/tmpq6vnkz_q/march_native.s /tmp/tmpq6vnkz_q/empty.c -march=native
Flags extracted: -mabi=lp64 -mcpu=cortex-a72+crc -mlittle-endian
Flags extracted: -mabi=lp64 -march= [..]
                            ^^^^^^^

Could you share the output of this command?

gcc -S -fverbose-asm -o /dev/stdout "$(mktemp --suffix=.c)" -march=native | grep -- -march

Thanks!

[jlpoolen]

As requested:

 pizero2 /var/tmp/portage/app-misc/resolve-march-native-2.2.0/temp # date; gcc -S -fverbose-asm -o /dev/stdout "$(mktemp --suffix=.c)" -march=native | grep -- -march
 Thu Oct 19 08:17:35 AM PDT 2023
 pizero2 /var/tmp/portage/app-misc/resolve-march-native-2.2.0/temp # 

[hartwork]

@jlpoolen so no output? I guess that means resolve-march-native will not be able to support that system then. Any ideas?

[jlpoolen]

I'm over-whelmed now dealing with electronics stuff now (I2C on Gentoo Linux for Raspberry Pi). I think the package in Gentoo should be marked as non-viable for arm64 until something comes to light in this Issue. That way people with arm64 platforms will not try to install it and then be confronted with the same problems I encountered. I'll keep it in mind, and make an update here if I find something useful. (I'm not a fan of Python, I remain true to Perl.)

I came across this tool because it was recommended in the Gentoo Binary Package page as I am going to set up a server for Raspberry Pi packages which are compiled on a beefier RPi4B(8MBs) but can be run on a Pi Zero 2 W and I thought marshalling all the possible flags an excellent approach.

[hartwork]

@jlpoolen can we guarantee that all arm64 platforms will show the same error as yours? Can you share gcc --version output? Trying with 1.0.0-r1 in Gentoo could also be of interest, it did less of what 2.2.0 does today.

[jlpoolen]

You have two requests: 1) gcc --version

jlpoole@pizero2 ~ $ gcc --version
gcc (Gentoo 13.2.1_p20230826 p7) 13.2.1 20230826
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

jlpoole@pizero2 ~ $ eix -I gcc
[I] sys-devel/gcc
     Available versions:  
     (8.5.0) [M]8.5.0-r1^t
     (9.5.0) [M]9.5.0^t
     (10)   10.4.1_p20230426-r1^t 10.5.0^t
     (11)   11.3.1_p20230427^t ~11.4.1_p20230622^t ~11.4.1_p20230824^t **11.4.1_p20230921^t **11.4.1_p20230928^t **11.4.1_p20231005^t **11.5.9999*l^t
     (12)   12.3.1_p20230526^t ~12.3.1_p20230825^t **12.3.1_p20230922^t **12.3.1_p20230929^t **12.3.1_p20231006^t **12.4.9999*l^t
     (13)   13.2.1_p20230826^t **13.2.1_p20230923^t **13.2.1_p20230930^t **13.2.1_p20231007^t **13.3.9999*l^t
     (14)   **14.0.0_pre20230917^t **14.0.0_pre20230924^t **14.0.0_pre20231001^t **14.0.0_pre20231008^t **14.0.0.9999*l^t
       {ada cet custom-cflags +cxx d debug default-stack-clash-protection default-znow doc fixed-point +fortran go graphite hardened ieee-long-double jit libssp lto modula2 mpx multilib +nls +nptl objc objc++ objc-gc (+)openmp +pch pgo +pie rust +sanitize +ssp systemtap test valgrind vanilla vtv zstd}
     Installed versions:  13.2.1_p20230826(13)^t(11:03:09 AM 10/12/2023)(cxx fortran nls nptl openmp pgo pie sanitize ssp zstd -ada -cet -custom-cflags -d -debug -default-stack-clash-protection -default-znow -doc -fixed-point -go -graphite -hardened -ieee-long-double -jit -libssp -lto -modula2 -multilib -objc -objc++ -objc-gc -pch -systemtap -test -valgrind -vanilla -vtv)
     Homepage:            https://gcc.gnu.org/
     Description:         The GNU Compiler Collection

[I] sys-devel/gcc-config
     Available versions:  2.11 **9999*l {+cc-wrappers +native-symlinks}
     Installed versions:  2.11(09:14:43 PM 08/16/2023)(cc-wrappers native-symlinks)
     Homepage:            https://gitweb.gentoo.org/proj/gcc-config.git/
     Description:         Utility to manage compilers

Found 2 matches
jlpoole@pizero2 ~ $ date
Thu Oct 19 12:14:29 PM PDT 2023
jlpoole@pizero2 ~ $ 

2) What does older version produce on an arm64 platform?

 Sorry, Gentoo's package for the 1.x build has a dependency of Python 3.10 which I currently do not have on my system, I only have 3.11 and installing 3.10 might create problems for me outside of this project.

     jlpoole@pizero2 ~ $ eix resolve-march-native
[I] app-misc/resolve-march-native
     Available versions:  (~)1.0.0-r1^t (~)2.1.0^t (~)2.1.0-r1^t (~)2.2.0^t{gpkg} {test PYTHON_TARGETS="python3_10 python3_11"}
     Installed versions:  2.2.0^t{gpkg}(08:19:32 AM 10/18/2023)(-test PYTHON_TARGETS="python3_11 -python3_10")
     Homepage:            https://github.com/hartwork/resolve-march-native
     Description:         Resolve GCC flag -march=native

jlpoole@pizero2 ~ $ emerge =app-misc/resolve-march-native-1.0.0-r1 -p

These are the packages that would be merged:

Calculating dependencies  

!!! Problem resolving dependencies for =app-misc/resolve-march-native-1.0.0-r1
... done!
Dependency resolution took 3.79 s.

!!! The ebuild selected to satisfy "=app-misc/resolve-march-native-1.0.0-r1" has unmet requirements.
- app-misc/resolve-march-native-1.0.0-r1::gentoo USE="-test" PYTHON_TARGETS="-python3_10"

  The following REQUIRED_USE flag constraints are unsatisfied:
    python_targets_python3_10

  The above constraints are a subset of the following complete expression:
    any-of ( python_targets_python3_10 )

jlpoole@pizero2 ~ $ 

[hartwork]

@jlpoolen thanks for the updates. Any ideas how to continue?

[jlpoolen]

I'll raise this issue with Mike Jones, the principal developer of the GenPi64 project and see what his thoughts are. It might be several days or weeks before I have anything further.

[jlpoolen]

For future reference: https://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html

[hartwork]

@jlpoolen any chance you could give pull request #107 a try? The description there includes commands to ease running code off the branch behind it. Thanks in advance!

[jlpoolen]

done, see https://github.com/hartwork/resolve-march-native/pull/107#issuecomment-1786267109

[jlpoolen]

Since this was closed, I tested and the results are below. Please let me know if there is something I should do rather than below to test? I see now that I do get a flag in non-debug mode, and in debug mode I get several flags, however, with errors.

jlpoole@pizero2 /usr/local/src/resolve-march-native $ cd resolve-march-native/
jlpoole@pizero2 /usr/local/src/resolve-march-native/resolve-march-native $ date
Sat Nov  4 08:02:52 AM PDT 2023
jlpoole@pizero2 /usr/local/src/resolve-march-native/resolve-march-native $ git pull
remote: Enumerating objects: 11, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 11 (delta 2), reused 3 (delta 2), pack-reused 8
Unpacking objects: 100% (11/11), 2.58 KiB | 330.00 KiB/s, done.
From https://github.com/hartwork/resolve-march-native
   1bfc091..905f971  master     -> origin/master
 * [new tag]         3.0.0      -> 3.0.0
Your configuration specifies to merge with the ref 'refs/heads/issue-78-support-mcpu'
from the remote, but no such ref was fetched.
jlpoole@pizero2 /usr/local/src/resolve-march-native/resolve-march-native $ cd ..
jlpoole@pizero2 /usr/local/src/resolve-march-native $ ls
resolve-march-native
jlpoole@pizero2 /usr/local/src/resolve-march-native $ mv resolve-march-native/ resolve-march-native/_TESTING
mv: cannot move 'resolve-march-native/' to a subdirectory of itself, 'resolve-march-native/_TESTING'
jlpoole@pizero2 /usr/local/src/resolve-march-native $ ls
resolve-march-native
jlpoole@pizero2 /usr/local/src/resolve-march-native $ mv resolve-march-native/ resolve-march-native_TESTING
jlpoole@pizero2 /usr/local/src/resolve-march-native $ git clone https://github.com/hartwork/resolve-march-native.git
Cloning into 'resolve-march-native'...
remote: Enumerating objects: 1068, done.
remote: Counting objects: 100% (130/130), done.
remote: Compressing objects: 100% (62/62), done.
remote: Total 1068 (delta 81), reused 68 (delta 68), pack-reused 938
Receiving objects: 100% (1068/1068), 192.95 KiB | 7.42 MiB/s, done.
Resolving deltas: 100% (622/622), done.
jlpoole@pizero2 /usr/local/src/resolve-march-native $ ls
resolve-march-native  resolve-march-native_TESTING
jlpoole@pizero2 /usr/local/src/resolve-march-native $ cd resolve-march-native
jlpoole@pizero2 /usr/local/src/resolve-march-native/resolve-march-native $ python3 -m venv venv  #
jlpoole@pizero2 /usr/local/src/resolve-march-native/resolve-march-native $ 
jlpoole@pizero2 /usr/local/src/resolve-march-native/resolve-march-native $ python3 -m venv venv 
jlpoole@pizero2 /usr/local/src/resolve-march-native/resolve-march-native $ source venv/bin/activate
(venv) jlpoole@pizero2 /usr/local/src/resolve-march-native/resolve-march-native $ pip install -e .
Obtaining file:///usr/local/src/resolve-march-native/resolve-march-native
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: resolve-march-native
  Building editable for resolve-march-native (pyproject.toml) ... done
  Created wheel for resolve-march-native: filename=resolve_march_native-3.0.0-0.editable-py3-none-any.whl size=4018 sha256=33ecd1f0eeb8563c434b8f97f955e8355cf4715235e86ced1bfcf1bb55a0e7f5
  Stored in directory: /tmp/pip-ephem-wheel-cache-a2d27j5z/wheels/cc/e0/0e/883e5c62eb05f661e2d5e515e8046eca5a9f8aa7d8822f7c7f
Successfully built resolve-march-native
Installing collected packages: resolve-march-native
Successfully installed resolve-march-native-3.0.0

[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip
(venv) jlpoole@pizero2 /usr/local/src/resolve-march-native/resolve-march-native $ hash resolve-march-native
(venv) jlpoole@pizero2 /usr/local/src/resolve-march-native/resolve-march-native $ date; resolve-march-native --debug
Sat Nov  4 08:08:33 AM PDT 2023
# gcc -S -fverbose-asm -o /tmp/tmpmidq2_zr/march_native.s /tmp/tmpmidq2_zr/empty.c -march=native
Flags extracted: -mabi=lp64 -mcpu=cortex-a72+crc -mlittle-endian
Flags extracted: -mabi=lp64 -march= -mbranch-protection= -mcmodel=small -mcpu=cortex-a72+crc -mfix-cortex-a53-835769 -mfix-cortex-a53-843419 -mglibc -mharden-sls= -mlittle-endian -momit-leaf-frame-pointer -moutline-atomics -moverride= -mpc-relative-literal-loads -msign-return-address=none -mstack-protector-guard-offset= -mstack-protector-guard-reg= -mstack-protector-guard=global -msve-vector-bits=scalable -mtls-dialect=desc -mtls-size=24 -mtune=
# gcc -S -fverbose-asm -o /tmp/tmpd0oovv0t/march_native.s /tmp/tmpd0oovv0t/empty.c -march=cortex-a72+crc
cc1: error: unknown value 'cortex-a72+crc' for '-march'
cc1: note: valid arguments are: armv8-a armv8.1-a armv8.2-a armv8.3-a armv8.4-a armv8.5-a armv8.6-a armv8.7-a armv8.8-a armv8-r armv9-a armv9.1-a armv9.2-a armv9.3-a native
cc1: note: did you mean '-mcpu=cortex-a72+crc'?
# gcc -S -fverbose-asm -o /tmp/tmplcotgw2q/march_native.s /tmp/tmplcotgw2q/empty.c -mcpu=cortex-a72+crc
Flags extracted: -mabi=lp64 -mcpu=cortex-a72+crc -mlittle-endian
cc1: error: unknown value 'cortex-a72+crc' for '-march'
cc1: note: valid arguments are: armv8-a armv8.1-a armv8.2-a armv8.3-a armv8.4-a armv8.5-a armv8.6-a armv8.7-a armv8.8-a armv8-r armv9-a armv9.1-a armv9.2-a armv9.3-a native
cc1: note: did you mean '-mcpu=cortex-a72+crc'?
Flags extracted: -mabi=lp64 -march= -mbranch-protection= -mcmodel=small -mcpu=cortex-a72+crc -mfix-cortex-a53-835769 -mfix-cortex-a53-843419 -mglibc -mharden-sls= -mlittle-endian -momit-leaf-frame-pointer -moutline-atomics -moverride= -mpc-relative-literal-loads -msign-return-address=none -mstack-protector-guard-offset= -mstack-protector-guard-reg= -mstack-protector-guard=global -msve-vector-bits=scalable -mtls-dialect=desc -mtls-size=24 -mtune=
-mcpu=cortex-a72+crc
(venv) jlpoole@pizero2 /usr/local/src/resolve-march-native/resolve-march-native $ date; resolve-march-native 
Sat Nov  4 08:15:46 AM PDT 2023
-mcpu=cortex-a72+crc
(venv) jlpoole@pizero2 /usr/local/src/resolve-march-native/resolve-march-native $

[hartwork]

@jlpoolen the error output is expected for debug mode for now. May go away later but all good for now, as far as I can see. Good?

[jlpoolen]

@jlpoolen the error output is expected for debug mode for now. May go away later but all good for now, as far as I can see. Good?

So the single flag returned in regular mode is expected, whereas the multiple flags listed in the debug mode are just tests/debug checking? I came to this tool because I want to create a binary server, so I referenced Gentoo's Binary [P]ackage [G]uide (under the section ""Handling *FLAGS in detail) suggesting I marshal all the variables.

The debug listing has:

Flags extracted: -mabi=lp64 -march= -mbranch-protection= -mcmodel=small -mcpu=cortex-a72+crc -mfix-cortex-a53-835769 -mfix-cortex-a53-843419 -mglibc -mharden-sls= -mlittle-endian -momit-leaf-frame-pointer -moutline-atomics -moverride= -mpc-relative-literal-loads -msign-return-address=none -mstack-protector-guard-offset= -mstack-protector-guard-reg= -mstack-protector-guard=global -msve-vector-bits=scalable -mtls-dialect=desc -mtls-size=24 -mtune=
-mcpu=cortex-a72+crc

whereas the regular mode listing just has one:

-mcpu=cortex-a72+crc

I confess I do not know much, if anything, about compiler flags. Nonetheless, the debug mode listing of several flags vs. the regular mode listing of one flag causes me to wonder if I would be missing something to rely only on the regular mode listing.

[hartwork]

@jlpoolen what resolve-march-native does it first figures out what value to use for march/mcpu and then queries GCC further which of the other flags are implied by that very architecture already and then cuts them away as redundant. If that's not what you want, you can always copy paste from the debug output of course and use the verbose long version.

[jlpoolen]

Version bumped to 3.0.0 to pick up changes: https://bugs.gentoo.org/916862

[thesamesam]

It's already in tree.

[jlpoolen]

How would I know that? I updated my tree from OSU and I checked the Bug database. I updated the Gentoo Bug asking the same question.

On Sat, Nov 4, 2023 at 10:50 AM Sam James @.***> wrote:

It's already in tree.

— Reply to this email directly, view it on GitHub https://github.com/hartwork/resolve-march-native/issues/105#issuecomment-1793506785, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABXP4L7IZJO2NJWV6WSSSDYCZ567AVCNFSM6AAAAAA6GJNRCOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJTGUYDMNZYGU . You are receiving this because you were mentioned.Message ID: @.***>

-- John L. Poole

707-812-1323 new email:* @. @.> old gmail.com http://gmail.com account is being phased out*

[hartwork]

How would I know that? I updated my tree from OSU and I checked the Bug database. I updated the Gentoo Bug asking the same question.

For anyone else reading, this is answered by @thesamesam at https://bugs.gentoo.org/916862#c3 .

@jlpoolen maybe https://github.com/gentoo-mirror/gentoo with the MD5 cache pre-rendered is what you want.