search

raspberrypi / picotool

BSD 3-Clause "New" or "Revised" License
592 stars 106 forks source link

elf2uf2 on s390x: ERROR: Not an ELF file #104

Open josch opened 4 months ago

josch commented 4 months ago

Hi,

I have this CMakeLists.txt:

cmake_minimum_required(VERSION 3.13)
include(pico_sdk_import.cmake)
project(hello_world)
pico_sdk_init()

add_executable(hello_world hello.c)
target_link_libraries(hello_world pico_stdlib)
pico_add_extra_outputs(hello_world)

and this hello-world program (yes this program is not even making use of the pico sdk but the problem is also there if we add that):

#include <stdio.h>

int main() {
    printf("Hello, world!\n");
    return 0;
}

On s390x the build will fail like this:

$ make VERBOSE=1
[...]
/usr/bin/arm-none-eabi-objcopy -Oihex /home/josch/hello_world.elf hello_world.hex
/usr/bin/arm-none-eabi-objcopy -Obinary /home/josch/hello_world.elf hello_world.bin
/usr/bin/arm-none-eabi-objdump -h /home/josch/hello_world.elf > hello_world.dis
/usr/bin/arm-none-eabi-objdump -d /home/josch/hello_world.elf >> hello_world.dis
elf2uf2/elf2uf2 /home/josch/hello_world.elf hello_world.uf2
ERROR: Not an ELF file
make[2]: *** [CMakeFiles/hello_world.dir/build.make:812: hello_world.elf] Error 254
make[2]: *** Deleting file 'hello_world.elf'
make[2]: Leaving directory '/home/josch'
make[1]: *** [CMakeFiles/Makefile2:1491: CMakeFiles/hello_world.dir/all] Error 2
make[1]: Leaving directory '/home/josch'
make: *** [Makefile:91: all] Error 2

I added some printfs into tools/elf2uf2/main.cpp to investigate the values of ELF_MAGIC and eh_out.common.magic in read_and_check_elf32_header and got this:

ELF_MAGIC = 464c457f
eh_out.common.magic = 7f454c46

There seems to be an endian problem somewhere? The file utility seems to think that my hello_world.elf has the correct magic:

$ file hello_world.elf
hello_world.elf: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, with debug_info, not stripped

Any ideas?

lurch commented 4 months ago

I guess this would be fixed by raspberrypi/pico-sdk#329 ?

josch commented 4 months ago

Thank you! In the mean-time I had this patch cooked up:

--- a/tools/elf2uf2/main.cpp    2023-06-13 22:27:33.000000000 +0000
+++ b/tools/elf2uf2/main.cpp    2024-06-05 08:20:56.156138030 +0000
@@ -11,6 +11,7 @@
 #include <cstring>
 #include <cstdarg>
 #include <algorithm>
+#include <endian.h>
 #include "boot/uf2.h"
 #include "elf.h"

@@ -101,6 +103,20 @@
     if (1 != fread(&eh_out, sizeof(eh_out), 1, in)) {
         return fail(ERROR_READ_FAILED, "Unable to read ELF header");
     }
+    eh_out.common.magic    = le32toh(eh_out.common.magic);
+    eh_out.common.type     = le16toh(eh_out.common.type);
+    eh_out.common.machine  = le16toh(eh_out.common.machine);
+    eh_out.common.version2 = le32toh(eh_out.common.version2);
+    eh_out.entry           = le32toh(eh_out.entry);
+    eh_out.ph_offset       = le32toh(eh_out.ph_offset);
+    eh_out.sh_offset       = le32toh(eh_out.sh_offset);
+    eh_out.flags           = le32toh(eh_out.flags);
+    eh_out.eh_size         = le16toh(eh_out.eh_size);
+    eh_out.ph_entry_size   = le16toh(eh_out.ph_entry_size);
+    eh_out.ph_num          = le16toh(eh_out.ph_num);
+    eh_out.sh_entry_size   = le16toh(eh_out.sh_entry_size);
+    eh_out.sh_num          = le16toh(eh_out.sh_num);
+    eh_out.sh_str_index    = le16toh(eh_out.sh_str_index);
     if (eh_out.common.magic != ELF_MAGIC) {
         return fail(ERROR_FORMAT, "Not an ELF file");
     }
@@ -154,6 +175,16 @@
         if (eh.ph_num != fread(&entries[0], sizeof(struct elf32_ph_entry), eh.ph_num, in)) {
             return fail_read_error();
         }
+   for (int i = 0; i < eh.ph_num; i++) {
+       entries[i].type = le32toh(entries[i].type);
+       entries[i].offset = le32toh(entries[i].offset);
+       entries[i].vaddr = le32toh(entries[i].vaddr);
+       entries[i].paddr = le32toh(entries[i].paddr);
+       entries[i].filez = le32toh(entries[i].filez);
+       entries[i].memsz = le32toh(entries[i].memsz);
+       entries[i].flags = le32toh(entries[i].flags);
+       entries[i].align = le32toh(entries[i].align);
+   }
     }
     return 0;
 }

I also rebased the patch from raspberrypi/pico-sdk#329 onto my local pico-sdk version and:

[100%] Linking CXX executable hello_world.elf
[100%] Built target hello_world

It works fine, thank you!

will-v-pi commented 2 months ago

Transfering this over to picotool, as that's where the elf2uf2 functionality is now

I believe that I have fixed this (in a different way to raspberrypi/pico-sdk#329) for the picotool uf2 convert command which replaces elf2uf2 - however I haven't tried fixing it for the rest of picotool as I don't have a big-endian system to test it on.

@josch Could you test the new picotool and see if this issue has actually been fixed for big-endian systems?

josch commented 1 month ago

Could you test the new picotool and see if this issue has actually been fixed for big-endian systems?

Thank you for working on this! I tried building the new pico-sdk and picotool 2.0 but there seems to be a build dependency loop here and I'm unsure how I am supposed to resolve it. When building pico-sdk 2.0 I get:

CMake Warning at tools/Findpicotool.cmake:28 (message):
  No installed picotool with version 2.0.0 found

And when trying to build picotool I get:

CMake Error at CMakeLists.txt:23 (message):
  Raspberry Pi Pico SDK version 2.0.0 (or later) required.

How am I as a distribution packager supposed to resolve this in practice? Thank you!

will-v-pi commented 1 month ago

Picotool doesn’t require a built SDK, it just requires some of the SDK header files, so you should be able to build picotool first and then build the SDK. I’m not familiar with packaging, but I hope that should be possible?

will-v-pi commented 1 month ago

Additionally, if you’re looking into packaging pico-sdk and picotool then I’d be happy to answer any other questions you have about them

josch commented 1 month ago

Picotool doesn’t require a built SDK, it just requires some of the SDK header files, so you should be able to build picotool first and then build the SDK. I’m not familiar with packaging, but I hope that should be possible?

But if building picotool requires SDK header files, then i need to build the SDK first to create a package containing these header files. But I cannot build the SDK without picotool.

If there is no way around this, then I will have to merge both source packages into one because either one cannot be build without having the source of the other available.

Additionally, if you’re looking into packaging pico-sdk and picotool then I’d be happy to answer any other questions you have about them

It's already packaged:

But with version 2.0 there seems to be a circular dependency between the two. Maybe there is some cmake flag i can pass to break this cycle?

will-v-pi commented 1 month ago

You can pass -DPICO_NO_PICOTOOL=1 to the SDK build, and it will skip anything that requires picotool, which for building the SDK is just the UF2s for the test programs. So that flag should be fine for just building the docs.

Also, looking at that package, you don’t currently have a pioasm binary - it might be helpful to build and install that (either as part of the sdk, or a separate package), which can be done by cding into the tools/pioasm directory, then doing a normal CMake build and install from there. Without the pioasm binary, each project using PIO will need to build the pioasm tool separately

josch commented 1 month ago

Thank you, I implemented both your suggestions. That was very helpful, thanks!

I'll upload pico-sdk 2.0 to experimental soon and am now stuck with also updating picotool to 2.0: https://github.com/raspberrypi/picotool/issues/139

josch commented 1 month ago

Additionally, if you’re looking into packaging pico-sdk and picotool then I’d be happy to answer any other questions you have about them

If you don't mind I'd like to abuse this issue for another packaging related question. Both pico-sdk as well as picotool are now in Debian unstable in version 2.0:

Now suppose somebody wants to use that package in their project. Lets look at this minimal reproducer:

cmake_minimum_required(VERSION 3.13)
include(/usr/src/pico-sdk/pico_sdk_init.cmake)
project(hello_world)
pico_sdk_init()

add_executable(hello_world hello.c)
target_link_libraries(hello_world pico_stdlib)
pico_add_extra_outputs(hello_world)
#include <stdio.h>
#include "pico/stdlib.h"

int main() {
    setup_default_uart();
    printf("Hello, world!\n");
    return 0;
}

But trying to build this will fail with:

CMake Warning at /usr/src/pico-sdk/tools/Findpicotool.cmake:28 (message):
  No installed picotool with version 2.0.0 found - building from source

  It is recommended to build and install picotool separately, or to set
  PICOTOOL_FETCH_FROM_GIT_PATH to a common directory for all your SDK
  projects
Call Stack (most recent call first):
  /usr/src/pico-sdk/tools/CMakeLists.txt:138 (find_package)
  /usr/src/pico-sdk/src/cmake/on_device.cmake:33 (pico_init_picotool)
  /usr/src/pico-sdk/src/rp2040/boot_stage2/CMakeLists.txt:57 (pico_add_dis_output)
  /usr/src/pico-sdk/src/rp2040/boot_stage2/CMakeLists.txt:101 (pico_define_boot_stage2)

So it cannot find picotool. The picotool Debian package installs the cmake files:

drwxr-xr-x root/root         0 2024-09-09 22:54 ./usr/lib/aarch64-linux-gnu/cmake/
drwxr-xr-x root/root         0 2024-09-09 22:54 ./usr/lib/aarch64-linux-gnu/cmake/picotool/
-rw-r--r-- root/root        96 2024-09-09 22:54 ./usr/lib/aarch64-linux-gnu/cmake/picotool/picotoolConfig.cmake
-rw-r--r-- root/root      2303 2024-09-09 22:54 ./usr/lib/aarch64-linux-gnu/cmake/picotool/picotoolConfigVersion.cmake
-rw-r--r-- root/root       745 2024-09-09 22:54 ./usr/lib/aarch64-linux-gnu/cmake/picotool/picotoolTargets-none.cmake
-rw-r--r-- root/root      4447 2024-09-09 22:54 ./usr/lib/aarch64-linux-gnu/cmake/picotool/picotoolTargets.cmake

So why does ./tools/Findpicotool.cmake fail to find the installed picotool and tries to build it from a git clone instead?

Thanks!

will-v-pi commented 1 month ago

Ah yes - you'll need commit https://github.com/raspberrypi/picotool/commit/6ad9c2352cf90b4abf183143e3cbd9ec933b9ad1 as well, which changes the install dir to /usr/lib/cmake... rather than /usr/lib/aarch64-linux-gnu/cmake... - because CMake find_package which we use to find picotool doesn't search the architecture-specifc libdirs for the picotoolConfig.cmake file (see #117)

You'll probably also want to take raspberrypi/pico-sdk#1865 as well, which applies the same fix to pioasm

josch commented 1 month ago

Aha! https://github.com/raspberrypi/pico-sdk/pull/1865 reminds me of a very similar fix that I already did to pico-sdk to fix the pioasm cmake installation path:

https://sources.debian.org/src/pico-sdk/2.0.0-1/debian/patches/pioasm.cmake.patch/

As you can see from that patch, I chose to let it install to ${CMAKE_INSTALL_DATAROOTDIR} which is share by default instead of hard-coding lib. I'm unsure what is correct. In Debian I see architecture-independent cmake files in both /usr/share/cmake as well as in /usr/lib/cmake. I don't know if there is a document somewhere that prescribes whether $prefix/lib or $prefix/share should be used for arch-indep cmake files?

In any case, I'll apply your patch and report back once everything is built. Thanks!

josch commented 1 month ago

Okay, no that picotool 2.0 and pico-sdk 2.0 are both finally in Debian and building on all architectures, lets get back to the original problem of s390x. Right now, the test script, which compiles a "hello world" program as I posted above, succeeds on all architectures except for s390x: https://ci.debian.net/packages/p/pico-sdk/testing/s390x/51562659/

103s [100%] Linking CXX executable hello_world.elf
103s ERROR: ELF File Read from 0x100000 with size 0x10000 exceeds the file size 0x58fdc
103s make[2]: *** [CMakeFiles/hello_world.dir/build.make:973: hello_world.elf] Error 254
103s make[2]: *** Deleting file 'hello_world.elf'
103s make[1]: *** [CMakeFiles/Makefile2:1713: CMakeFiles/hello_world.dir/all] Error 2
103s make: *** [Makefile:91: all] Error 2

I can have a closer look on real s390x hardware later, but maybe you already have some theories that I could check?

Not sure if this is picotool or pico-sdk related?

josch commented 1 month ago

Ah yes, it fails in the picotool uf2 convert step. Here is the make VERBOSE=1 output:

/usr/bin/cmake -E cmake_link_script CMakeFiles/hello_world.dir/link.txt --verbose=1
/usr/bin/arm-none-eabi-g++ -mcpu=cortex-m0plus -mthumb -g -O3 -DNDEBUG -Wl,--build-id=none -Wl,-Map=hello_world.elf.map --specs=nosys.specs -Wl,--wrap=__clzsi2 -Wl,--wrap=__clzdi2 -Wl,--wrap=__ctzsi2 -Wl,--wrap=__popcountsi2 -Wl,--wrap=__popcountdi2 -Wl,--wrap=__clz -Wl,--wrap=__clzl -Wl,--wrap=__clzll -Wl,--wrap=__ctzdi2 -Wl,--wrap=__aeabi_idiv -Wl,--wrap=__aeabi_idivmod -Wl,--wrap=__aeabi_ldivmod -Wl,--wrap=__aeabi_uidiv -Wl,--wrap=__aeabi_uidivmod -Wl,--wrap=__aeabi_uldivmod -Wl,--wrap=__aeabi_dadd -Wl,--wrap=__aeabi_ddiv -Wl,--wrap=__aeabi_dmul -Wl,--wrap=__aeabi_drsub -Wl,--wrap=__aeabi_dsub -Wl,--wrap=__aeabi_cdcmpeq -Wl,--wrap=__aeabi_cdrcmple -Wl,--wrap=__aeabi_cdcmple -Wl,--wrap=__aeabi_dcmpeq -Wl,--wrap=__aeabi_dcmplt -Wl,--wrap=__aeabi_dcmple -Wl,--wrap=__aeabi_dcmpge -Wl,--wrap=__aeabi_dcmpgt -Wl,--wrap=__aeabi_dcmpun -Wl,--wrap=__aeabi_i2d -Wl,--wrap=__aeabi_l2d -Wl,--wrap=__aeabi_ui2d -Wl,--wrap=__aeabi_ul2d -Wl,--wrap=__aeabi_d2iz -Wl,--wrap=__aeabi_d2lz -Wl,--wrap=__aeabi_d2uiz -Wl,--wrap=__aeabi_d2ulz -Wl,--wrap=__aeabi_d2f -Wl,--wrap=sqrt -Wl,--wrap=cos -Wl,--wrap=sin -Wl,--wrap=tan -Wl,--wrap=atan2 -Wl,--wrap=exp -Wl,--wrap=log -Wl,--wrap=ldexp -Wl,--wrap=copysign -Wl,--wrap=trunc -Wl,--wrap=floor -Wl,--wrap=ceil -Wl,--wrap=round -Wl,--wrap=sincos -Wl,--wrap=asin -Wl,--wrap=acos -Wl,--wrap=atan -Wl,--wrap=sinh -Wl,--wrap=cosh -Wl,--wrap=tanh -Wl,--wrap=asinh -Wl,--wrap=acosh -Wl,--wrap=atanh -Wl,--wrap=exp2 -Wl,--wrap=log2 -Wl,--wrap=exp10 -Wl,--wrap=log10 -Wl,--wrap=pow -Wl,--wrap=powint -Wl,--wrap=hypot -Wl,--wrap=cbrt -Wl,--wrap=fmod -Wl,--wrap=drem -Wl,--wrap=remainder -Wl,--wrap=remquo -Wl,--wrap=expm1 -Wl,--wrap=log1p -Wl,--wrap=fma -Wl,--wrap=__aeabi_lmul -Wl,--wrap=__aeabi_fadd -Wl,--wrap=__aeabi_fdiv -Wl,--wrap=__aeabi_fmul -Wl,--wrap=__aeabi_frsub -Wl,--wrap=__aeabi_fsub -Wl,--wrap=__aeabi_cfcmpeq -Wl,--wrap=__aeabi_cfrcmple -Wl,--wrap=__aeabi_cfcmple -Wl,--wrap=__aeabi_fcmpeq -Wl,--wrap=__aeabi_fcmplt -Wl,--wrap=__aeabi_fcmple -Wl,--wrap=__aeabi_fcmpge -Wl,--wrap=__aeabi_fcmpgt -Wl,--wrap=__aeabi_fcmpun -Wl,--wrap=__aeabi_i2f -Wl,--wrap=__aeabi_l2f -Wl,--wrap=__aeabi_ui2f -Wl,--wrap=__aeabi_ul2f -Wl,--wrap=__aeabi_f2iz -Wl,--wrap=__aeabi_f2lz -Wl,--wrap=__aeabi_f2uiz -Wl,--wrap=__aeabi_f2ulz -Wl,--wrap=__aeabi_f2d -Wl,--wrap=sqrtf -Wl,--wrap=cosf -Wl,--wrap=sinf -Wl,--wrap=tanf -Wl,--wrap=atan2f -Wl,--wrap=expf -Wl,--wrap=logf -Wl,--wrap=ldexpf -Wl,--wrap=copysignf -Wl,--wrap=truncf -Wl,--wrap=floorf -Wl,--wrap=ceilf -Wl,--wrap=roundf -Wl,--wrap=sincosf -Wl,--wrap=asinf -Wl,--wrap=acosf -Wl,--wrap=atanf -Wl,--wrap=sinhf -Wl,--wrap=coshf -Wl,--wrap=tanhf -Wl,--wrap=asinhf -Wl,--wrap=acoshf -Wl,--wrap=atanhf -Wl,--wrap=exp2f -Wl,--wrap=log2f -Wl,--wrap=exp10f -Wl,--wrap=log10f -Wl,--wrap=powf -Wl,--wrap=powintf -Wl,--wrap=hypotf -Wl,--wrap=cbrtf -Wl,--wrap=fmodf -Wl,--wrap=dremf -Wl,--wrap=remainderf -Wl,--wrap=remquof -Wl,--wrap=expm1f -Wl,--wrap=log1pf -Wl,--wrap=fmaf -Wl,--wrap=malloc -Wl,--wrap=calloc -Wl,--wrap=realloc -Wl,--wrap=free -Wl,--wrap=memcpy -Wl,--wrap=memset -Wl,--wrap=__aeabi_memcpy -Wl,--wrap=__aeabi_memset -Wl,--wrap=__aeabi_memcpy4 -Wl,--wrap=__aeabi_memset4 -Wl,--wrap=__aeabi_memcpy8 -Wl,--wrap=__aeabi_memset8 -Wl,-L/home/josch/build -Wl,--script=/usr/src/pico-sdk/src/rp2_common/pico_crt0/rp2040/memmap_default.ld -Wl,-z,max-page-size=4096 -Wl,--gc-sections -Wl,--no-warn-rwx-segments -Wl,--wrap=sprintf -Wl,--wrap=snprintf -Wl,--wrap=vsnprintf -Wl,--wrap=printf -Wl,--wrap=vprintf -Wl,--wrap=puts -Wl,--wrap=putchar -Wl,--wrap=getchar CMakeFiles/hello_world.dir/hello.c.obj "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_stdlib/stdlib.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_gpio/gpio.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2040/pico_platform/platform.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_platform_panic/panic.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/common/hardware_claim/claim.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_sync/sync.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_sync_spin_lock/sync_spin_lock.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_irq/irq.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_irq/irq_handler_chain.S.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/common/pico_sync/sem.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/common/pico_sync/lock_core.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/common/pico_sync/mutex.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/common/pico_sync/critical_section.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/common/pico_time/time.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/common/pico_time/timeout_helper.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_timer/timer.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/common/pico_util/datetime.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/common/pico_util/pheap.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/common/pico_util/queue.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_uart/uart.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_clocks/clocks.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_pll/pll.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_vreg/vreg.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_watchdog/watchdog.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_ticks/ticks.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_xosc/xosc.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_divider/divider.S.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_runtime/runtime.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_runtime_init/runtime_init.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_runtime_init/runtime_init_clocks.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_runtime_init/runtime_init_stack_guard.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_bit_ops/bit_ops_aeabi.S.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_bootrom/bootrom.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_bootrom/bootrom_lock.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/hardware_boot_lock/boot_lock.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_divider/divider_hardware.S.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_double/double_aeabi_rp2040.S.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_double/double_init_rom_rp2040.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_double/double_math.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_double/double_v1_rom_shim_rp2040.S.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_int64_ops/pico_int64_ops_aeabi.S.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_float/float_aeabi_rp2040.S.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_float/float_init_rom_rp2040.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_float/float_math.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_float/float_v1_rom_shim_rp2040.S.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_malloc/malloc.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_mem_ops/mem_ops_aeabi.S.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_atomic/atomic.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_cxx_options/new_delete.cpp.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_standard_binary_info/standard_binary_info.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_printf/printf.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_crt0/crt0.S.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_clib_interface/newlib_interface.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_stdio/stdio.c.obj" "CMakeFiles/hello_world.dir/usr/src/pico-sdk/src/rp2_common/pico_stdio_uart/stdio_uart.c.obj" -o hello_world.elf  pico-sdk/src/rp2040/boot_stage2/bs2_default_padded_checksummed.S
/usr/bin/arm-none-eabi-objdump -h /home/josch/build/hello_world.elf > hello_world.dis
/usr/bin/arm-none-eabi-objdump -d /home/josch/build/hello_world.elf >> hello_world.dis
/usr/bin/picotool coprodis --quiet hello_world.dis hello_world.dis || /usr/bin/cmake -E echo "WARNING: Disassembly is not correct"
/usr/bin/arm-none-eabi-objcopy -Obinary /home/josch/build/hello_world.elf hello_world.bin
/usr/bin/picotool uf2 convert --quiet /home/josch/build/hello_world.elf hello_world.uf2 --family rp2040 --abs-block
ERROR: ELF File Read from 0x100000 with size 0x10000 exceeds the file size 0x58f1c
make[2]: *** [CMakeFiles/hello_world.dir/build.make:973: hello_world.elf] Error 254
make[2]: *** Deleting file 'hello_world.elf'
make[2]: Leaving directory '/home/josch/build'
make[1]: *** [CMakeFiles/Makefile2:1713: CMakeFiles/hello_world.dir/all] Error 2
make[1]: Leaving directory '/home/josch/build'
make: *** [Makefile:91: all] Error 2
lurch commented 1 month ago

Random guess (I know nothing about "exotic" architectures or OSes), but could this be related to #133 ?

will-v-pi commented 1 month ago

Could you try setting PICO_NO_UF2 in CMake and then manually running /usr/bin/picotool uf2 convert --verbose /home/josch/build/hello_world.elf hello_world.uf2 --family rp2040 --abs-block - by default the sdk runs it with --quiet, which doesn't give much useful output

josch commented 1 month ago

Could you try setting PICO_NO_UF2 in CMake

You mean like this?

cmake_minimum_required(VERSION 3.13)
set(PICO_NO_UF2 ON)

include(pico_sdk_import.cmake)
project(hello_world)
pico_sdk_init()

add_executable(hello_world hello.c)
target_link_libraries(hello_world pico_stdlib)
pico_add_extra_outputs(hello_world)

With that setting, the build will succeed but i guess that's because PICO_NO_UF2 skips running picotool? That's probably because otherwise, make would remove hello_world.elf after it failed? In any case, here is what happens when I run picotool with --verbose after make:

$ /usr/bin/picotool uf2 convert --verbose /home/josch/build/hello_world.elf hello_world.uf2 --family rp2040 --abs-block
RP2350-E9: Adding absolute block to UF2 targeting 0x10ffff00
ERROR: ELF File Read from 0x100000 with size 0x10000 exceeds the file size 0x58f1c
will-v-pi commented 1 month ago

Can you try this patch to picotool?

diff --git a/elf/elf_file.cpp b/elf/elf_file.cpp
index 08988a1..551b188 100644
--- a/elf/elf_file.cpp
+++ b/elf/elf_file.cpp
@@ -210,14 +210,14 @@ void elf_file::flatten(void) {

     elf_bytes.resize(std::max(eh.ph_offset + sizeof(elf32_ph_entry) * eh.ph_num, elf_bytes.size()));
     auto ph_entries_out = ph_entries;
-    for (auto ph : ph_entries_out) {
+    for (auto &ph : ph_entries_out) {
         ph_le(ph);  // swap to LE for writing
     }
     memcpy(&elf_bytes[eh.ph_offset], &ph_entries_out[0], sizeof(elf32_ph_entry) * eh.ph_num);

     elf_bytes.resize(std::max(eh.sh_offset + sizeof(elf32_sh_entry) * eh.sh_num, elf_bytes.size()));
     auto sh_entries_out = sh_entries;
-    for (auto sh : sh_entries_out) {
+    for (auto &sh : sh_entries_out) {
         sh_le(sh);  // swap to LE for writing
     }
     memcpy(&elf_bytes[eh.sh_offset], &sh_entries_out[0], sizeof(elf32_sh_entry) * eh.sh_num);
@@ -245,7 +245,7 @@ void elf_file::read_sh(void) {
     if (eh.sh_num) {
         sh_entries.resize(eh.sh_num);
         read_bytes(eh.sh_offset, sizeof(elf32_sh_entry) * eh.sh_num, &sh_entries[0]);
-        for (auto sh : sh_entries) {
+        for (auto &sh : sh_entries) {
             sh_he(sh);  // swap to Host for processing
         }
     }
@@ -356,7 +356,7 @@ void elf_file::read_ph(void) {
     if (eh.ph_num) {
         ph_entries.resize(eh.ph_num);
         read_bytes(eh.ph_offset, sizeof(elf32_ph_entry) * eh.ph_num, &ph_entries[0]);
-        for (auto ph : ph_entries) {
+        for (auto &ph : ph_entries) {
             ph_he(ph);  // swap to Host for processing
         }
     }
josch commented 1 month ago

Can you try this patch to picotool?

Thank you, with that I get:

$ ~/picotool-2.0.0/debian/picotool/usr/bin/picotool uf2 convert --verbose /home/josch/build/hello_world.elf hello_world.uf2 --family rp2040 --abs-block
RP2350-E9: Adding absolute block to UF2 targeting 0x10ffff00
$ echo $?
0

So I guess success? I'll add that patch to the Debian package and upload it again so that it can run on the CI servers. Thank you!

will-v-pi commented 1 month ago

It looks like more is required - that patch fixes the ELF handling, but the UF2 is being written with big-endian ints when they should be little.

I've got a docker container up and running with qemu emulation of s390x, so I can now test big-endian handling and will see if I can get something working

will-v-pi commented 1 month ago

I've got a it generating a working UF2 on the big-endian branch - commits ecf378108c798387d0fc6f99bc6c2cad7bea11a6 and 7b9fc5e365b994afde75c393318c8aca40d7657f

I'll leave it there for now, as it at least lets you build a working UF2 with picotool and pico-sdk

josch commented 1 month ago

I've got a it generating a working UF2 on the big-endian branch - commits ecf3781 and 7b9fc5e

I'll leave it there for now, as it at least lets you build a working UF2 with picotool and pico-sdk

Thank you! With the patch you pasted 12 hours ago, running make already succeeds but I guess the result is broken? Is there an automated way to test for that? I'd like to add that test as part of my CI test.