search

3246251196 / adtools_testing

1 stars 1 forks source link

Prefix build is not finding required so files #4

Closed josefwegner closed 1 year ago

josefwegner commented 1 year ago

When building a cross compiler with the -p (prefix option), the test build can only finds "libm.so", not the other needed SOs (libc.so, libstdc++.so, ...).

How to reproduce:

  1. ./adt -c
  2. rm *.lha
  3. ./adt -e beta10 -m adtools_mod_clib2_June_2023 -p clib2 -b
  4. ./adt -p clib2 -t

Result:

$ ack 'Needed SO' tests
tests/1_rjd_test_example/log_test_1_rjd_test_example_clib2_DYNAMIC.txt
380:Needed SO, "libstdc++.so" NOT FOUND
381:Needed SO, "libm.so" FOUND
382:Needed SO, "libgcc.so" NOT FOUND
383:Needed SO, "libc.so" NOT FOUND

tests/1_rjd_test_example/log_test_1_rjd_test_example_newlib_DYNAMIC.txt
377:Needed SO, "libstdc++.so" NOT FOUND
378:Needed SO, "libgcc.so" NOT FOUND
379:Needed SO, "libc.so" NOT FOUND

tests/0_rjd_simplest_example/log_test_0_rjd_simplest_example_clib2_DYNAMIC.txt
334:Needed SO, "libgcc.so" NOT FOUND
335:Needed SO, "libc.so" NOT FOUND

tests/0_rjd_simplest_example/log_test_0_rjd_simplest_example_newlib_DYNAMIC.txt
332:Needed SO, "libgcc.so" NOT FOUND
333:Needed SO, "libc.so" NOT FOUND

tests/4_adtools_test_test-exceptions/log_test_4_adtools_test_test-exceptions_newlib_DYNAMIC.txt
377:Needed SO, "libstdc++.so" NOT FOUND
378:Needed SO, "libgcc.so" NOT FOUND
379:Needed SO, "libc.so" NOT FOUND

tests/4_adtools_test_test-exceptions/log_test_4_adtools_test_test-exceptions_clib2_DYNAMIC.txt
380:Needed SO, "libstdc++.so" NOT FOUND
381:Needed SO, "libm.so" FOUND
382:Needed SO, "libgcc.so" NOT FOUND
383:Needed SO, "libc.so" NOT FOUND

tests/8_adtools_test_test-call-once/log_test_8_adtools_test_test-call-once_newlib_DYNAMIC.txt
377:Needed SO, "libstdc++.so" NOT FOUND
378:Needed SO, "libgcc.so" NOT FOUND
379:Needed SO, "libc.so" NOT FOUND

tests/8_adtools_test_test-call-once/log_test_8_adtools_test_test-call-once_clib2_DYNAMIC.txt
380:Needed SO, "libstdc++.so" NOT FOUND
381:Needed SO, "libm.so" FOUND
382:Needed SO, "libgcc.so" NOT FOUND
383:Needed SO, "libc.so" NOT FOUND

tests/common.mak
74:                             echo "Needed SO, \"$${SO}\" FOUND" >> $(LOG_FILE) ; } ||               \
75:                             echo "Needed SO, \"$${SO}\" NOT FOUND" >> $(LOG_FILE) ;                \
78:                             echo "Needed SO, \"$${SO}\" FOUND" >> $(LOG_FILE) ; } ||               \
79:                             echo "Needed SO, \"$${SO}\" NOT FOUND" >> $(LOG_FILE) ;                \

tests/5_adtools_test_test-std-string/log_test_5_adtools_test_test-std-string_newlib_DYNAMIC.txt
377:Needed SO, "libstdc++.so" NOT FOUND
378:Needed SO, "libgcc.so" NOT FOUND
379:Needed SO, "libc.so" NOT FOUND

tests/5_adtools_test_test-std-string/log_test_5_adtools_test_test-std-string_clib2_DYNAMIC.txt
380:Needed SO, "libstdc++.so" NOT FOUND
381:Needed SO, "libm.so" FOUND
382:Needed SO, "libgcc.so" NOT FOUND
383:Needed SO, "libc.so" NOT FOUND

tests/2_capehill_adtools_issue_139_test_code/log_test_2_capehill_adtools_issue_139_test_code_newlib_DYNAMIC.txt
398:Needed SO, "libf_test_2_capehill_adtools_issue_139_test_code_newlib_DYNAMIC.so" FOUND
399:Needed SO, "libgcc.so" NOT FOUND
400:Needed SO, "libc.so" NOT FOUND

tests/2_capehill_adtools_issue_139_test_code/log_test_2_capehill_adtools_issue_139_test_code_clib2_DYNAMIC.txt
400:Needed SO, "libf_test_2_capehill_adtools_issue_139_test_code_clib2_DYNAMIC.so" FOUND
401:Needed SO, "libgcc.so" NOT FOUND
402:Needed SO, "libc.so" NOT FOUND

tests/7_adtools_test_test-whcar/log_test_7_adtools_test_test-whcar_newlib_DYNAMIC.txt
377:Needed SO, "libstdc++.so" NOT FOUND
378:Needed SO, "libgcc.so" NOT FOUND
379:Needed SO, "libc.so" NOT FOUND

tests/7_adtools_test_test-whcar/log_test_7_adtools_test_test-whcar_clib2_DYNAMIC.txt
380:Needed SO, "libstdc++.so" NOT FOUND
381:Needed SO, "libm.so" FOUND
382:Needed SO, "libgcc.so" NOT FOUND
383:Needed SO, "libc.so" NOT FOUND

tests/3_adtools_test_test-dynld/log_test_3_adtools_test_test-dynld_clib2_DYNAMIC.txt
377:Needed SO, "libstdc++.so" NOT FOUND
378:Needed SO, "libm.so" FOUND
379:Needed SO, "libgcc.so" NOT FOUND
380:Needed SO, "libc.so" NOT FOUND

tests/3_adtools_test_test-dynld/log_test_3_adtools_test_test-dynld_newlib_DYNAMIC.txt
374:Needed SO, "libstdc++.so" NOT FOUND
375:Needed SO, "libgcc.so" NOT FOUND
376:Needed SO, "libc.so" NOT FOUND

tests/9_rjd_math_linking/log_test_9_rjd_math_linking_newlib_DYNAMIC.txt
377:Needed SO, "libstdc++.so" NOT FOUND
378:Needed SO, "libgcc.so" NOT FOUND
379:Needed SO, "libc.so" NOT FOUND

tests/9_rjd_math_linking/log_test_9_rjd_math_linking_clib2_DYNAMIC.txt
380:Needed SO, "libstdc++.so" NOT FOUND
381:Needed SO, "libm.so" FOUND
382:Needed SO, "libgcc.so" NOT FOUND
383:Needed SO, "libc.so" NOT FOUND

tests/6_adtools_test_test-thread/log_test_6_adtools_test_test-thread_clib2_DYNAMIC.txt
384:Needed SO, "libstdc++.so" NOT FOUND
385:Needed SO, "libm.so" FOUND
386:Needed SO, "libgcc.so" NOT FOUND
387:Needed SO, "libc.so" NOT FOUND

tests/6_adtools_test_test-thread/log_test_6_adtools_test_test-thread_newlib_DYNAMIC.txt
381:Needed SO, "libstdc++.so" NOT FOUND
382:Needed SO, "libgcc.so" NOT FOUND
383:Needed SO, "libc.so" NOT FOUND

Expected: SOs are found

More Info: ./adt -x -t is also missing the SOs

3246251196 commented 1 year ago

@josefwegner Which operating system are you using on your RPi 4?

EDIT: Actually, you mentioned ARCH64 in another issue report.

3246251196 commented 1 year ago

@josefwegner Can you apply the following patch and run it after cleaning tests, capture the output and report?

applyMe.patch.txt

[ GITHUB will not allow me to upload with a .patch extension ]]

josefwegner commented 1 year ago

I am running Raspberry Pi OS aarch64/arm64 (based on Debian Bullseye):

$ cat /etc/*release*
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
$ uname -a
Linux raspberrypi 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr  3 17:24:16 BST 2023 aarch64 GNU/Linux
$ zsh --version
zsh 5.8 (aarch64-unknown-linux-gnu)
$ bash --version
GNU bash, Version 5.1.4(1)-release (aarch64-unknown-linux-gnu)
Copyright (C) 2020 Free Software Foundation, Inc.
Lizenz GPLv3+: GNU GPL Version 3 oder jünger <http://gnu.org/licenses/gpl.html>

Dies ist freie Software. Sie darf verändert und verteilt werden.
Es wird keine Garantie gewährt, soweit das Gesetz es zulässt.
3246251196 commented 1 year ago

Thanks for the information. I still wonder if you can try to apply that patch I added and get the output at some point, but, I should be able to reproduce this since I have a RPi4 and ROS also. I will check this later.

3246251196 commented 1 year ago

I tried to reproduce this.

uname -a
Linux raspberrypi 5.15.84-v8+ #1613 SMP PREEMPT Thu Jan 5 12:03:08 GMT 2023 aarch64 GNU/Linux
cat /etc/*release*
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
bash --version
GNU bash, version 5.1.4(1)-release (aarch64-unknown-linux-gnu)
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

And I cannot (see below) reproduce it. I tried to clean and create tests in a loop and I cannot find an occurrence of 'NOT FOUND'.

I build with:

cat .adt_session.cache
use_this_adtools="0"
gcc_ver="11"
binutils_ver="2.23.2"
expclib2_branch="beta10"
session_prefix="myPrefix"
gcc_path=""
adtools_repo_loc="https://github.com/sba1/adtools.git"
adtools_repo_branch=""

# ADTOOLS_MOD_SCRIPT="adtools_mod_clib2_June_2023"
EXP_CLIB2="1"
CLIB2_SHA1="beta10"
SDK_VERSION="54.16"
CROSS_PREFIX="/home/rjd/adtools_testing/myPrefixbuild/adt_build"

export EXP_CLIB2
export CLIB2_SHA1
export SDK_VERSION
export CROSS_PREFIX

Which took over 2 hours!

===

I said above to read below (here). Strangely, I did actually find an occurrence of it, but now I cannot reproduce it - even in the loop of 10 times I only see "FOUND"!

It would be good to see if you can consistently do it, and if you can it would be nice if you could not touch that environment so that we can try and track this down. Can you still consider providing me the STDOUT/STDERR of performing a TEST action so I can see that log. It may also be an idea to remove STDERR redirection to /dev/null (in the makefile / common.mak) so that we can see any errors that occur.

This is an interesting one! @josefwegner

3246251196 commented 1 year ago

Actually, I wonder if it is because of (btw, this makefile-ery was done before I thought of a need for prefixes):

# Unfortunately, the compiler libraries for newlib are not in a folder
# named newlib. For example, libgcc.so is inside:
# "lib/gcc/ppc-amigaos/11.3.0/libgcc.so", unlike clib which is in
# clib2.  GREP_OPT_C_LIB is a hack to get around this. First, assume
# that for any library we are looking for, we will find it in a
# location that contains the string of the particular C library being
# used. Then, in the case that we are newlib, so long as we remove
# anything that contains "clib2", we assume that is the newlib
# library. This works for the case that newlib is in the string and is
# not.
GREP_OPT_C_LIB=$(C_LIB)
ifeq ($(C_LIB),newlib)
    GREP_OPT_C_LIB=-v clib2
endif

and the fact that you called your prefix "clib2" - grep may then be stripping things away due to the unexpected clib2 prefix name - but it is a bit muddy and too late to think straight right now. Let me try - precisely - your reproduction steps and see where we get.

This is used in:

        LOC=$$(find $${CROSS_PREFIX} -name "$${SO}"            |                          \
                        grep $(GREP_OPT_C_LIB) |                          \
                        grep "ppc-amigaos")    ;                          \

I will set off another run have update tomorrow.

josefwegner commented 1 year ago

Good catch. The prefix could be the issue. I was in fact using clib2 as prefix when I had the issue with the missing so-files. For the last tests I used the prefix beta10 and v.1.0.0 instead.

3246251196 commented 1 year ago

@josefwegner I believe I have now made the fix. By using "clib" I was able to reproduce the problem on my main machine.

If this works for you, can you report back and close the ticket.

Thanks.

josefwegner commented 1 year ago

@3246251196: Sorry, using prefix clib2 still triggers the issues for me. Even with your fix (https://github.com/josefwegner/adtools_testing/commit/80f2baabbcd7d2d6e317ae7fe9e28707546ead43)

rm -Rf clib2* .adt*
./adt -c
./adt -e beta10 -m adtools_mod_clib2_June_2023 -p clib2 -b
./adt -p clib2 -t
...
### INFO: Session was cached to file "/home/pi/Projects/amigalocal/adtools_testing/.adt_session.cache". If there was a previous cache file it is now called: "/home/pi/Projects/amigalocal/adtools_testing/.adt_session.cache.bak"
### INFO: Cached settings:
### INFO:
### INFO: Settings Begin
### INFO: >>> use_this_adtools="0"
### INFO: >>> gcc_ver="11"
### INFO: >>> binutils_ver="2.23.2"
### INFO: >>> expclib2_branch="beta10"
### INFO: >>> session_prefix="clib2"
### INFO: >>> gcc_path=""
### INFO: >>> adtools_repo_loc="https://github.com/sba1/adtools.git"
### INFO: >>> adtools_repo_branch=""
### INFO: >>>
### INFO: >>> # ADTOOLS_MOD_SCRIPT="adtools_mod_clib2_June_2023"
### INFO: >>> EXP_CLIB2="1"
### INFO: >>> CLIB2_SHA1="beta10"
### INFO: >>> SDK_VERSION="54.16"
### INFO: >>> CROSS_PREFIX="/home/pi/Projects/amigalocal/adtools_testing/clib2build/adt_build"
### INFO: >>>
### INFO: >>> export EXP_CLIB2
### INFO: >>> export CLIB2_SHA1
### INFO: >>> export SDK_VERSION
### INFO: >>> export CROSS_PREFIX
### INFO: Settings End
### INFO:
grep "Needed SO" tests/9_rjd_math_linking/log_test_9_rjd_math_linking_clib2_DYNAMIC.txt
Needed SO, "libstdc++.so" NOT FOUND
Needed SO, "libm.so" FOUND
Needed SO, "libgcc.so" NOT FOUND
Needed SO, "libc.so" NOT FOUND
3246251196 commented 1 year ago

@josefwegner Perfect. I can still reproduce it too. I was too confident in thinking the change I made would work. So long as I can reproduce I can solve it. Give me some time. In the meantime, please refrain from using "clib2" as a prefix.

Thanks.

3246251196 commented 1 year ago

@josefwegner Can you try again with - at least - commit: df9214d0a9cf3788e5dd9f785721a7751d5a12d2 (https://github.com/3246251196/adtools_testing/commit/df9214d0a9cf3788e5dd9f785721a7751d5a12d2)

josefwegner commented 1 year ago

Yes! Thank you. With the last commit, the tests created with prefix clib2 contain all needed SO files.

3246251196 commented 1 year ago

Thanks. Re-open if you find some bug related to this again. Or just create a new ticket.