search

zephyrproject-rtos / zephyr

Primary Git Repository for the Zephyr Project. Zephyr is a new generation, scalable, optimized, secure RTOS for multiple hardware architectures.
https://docs.zephyrproject.org
Apache License 2.0
10.61k stars 6.5k forks source link

File linker-defs.h does not have a path to file offsets.h. #67268

Open paultino opened 9 months ago

paultino commented 9 months ago

Describe the bug

When I tried to build an application using the T2 topology, I received an error. When I specified the full path to the offsets.h file, the application was built successfully.

To Reproduce

  1. See error Screenshot 2024-01-06 164351

Expected behavior specify the path as indicated in the lines above in the linker-defs.h file.

Environment (please complete the following information):

github-actions[bot] commented 9 months ago

Hi @paultino! We appreciate you submitting your first issue for our open-source project. 🌟

Even though I'm a bot, I can assure you that the whole community is genuinely grateful for your time and effort. 🤖💙

MaureenHelm commented 8 months ago

Will you please provide more details about how to reproduce?

hellesvik-nordic commented 8 months ago

Several people reporting the same error over at DevZone: https://devzone.nordicsemi.com/f/nordic-q-a/106046/offsets-h-no-such-file-or-directory. I have not yet been able to figure out what causes this. A mentioned workaround is to tell west to use make instead of ninja. Also some people mention that the error is apparent in the first build, but that it works again on a subsequent builds.

hellesvik-nordic commented 8 months ago

Update: One of the users in the devzone ticket fixed this by including "#include <zephyr/drivers/i2c.h>" into the .cpp file instead of header file. Other users confirm that this fix worked.

paultino commented 8 months ago

I got the same thing yesterday on a fresh ncs installation.

C:/ncs/v2.5.1/zephyr/include/zephyr/linker/linker-defs.h:26:10: fatal error: offsets.h: No such file or directory 26 | #include | ^~~ compilation terminated.

I fixed it in both cases with this line:

include <../subsys/testsuite/ztest/include/zephyr/offsets.h>

Building a free-standing (T3) application. Using the lvgl example, I copied it to another folder. I didn't add any my code. I added my own shield instead of an 3_5_buydisplay due to the difference in the touch controller.

Upd. @hellesvik-nordic I don't use I2C, just the SPI.

C:/ncs/v2.5.1/zephyr/include/zephyr/linker/linker-defs.h:26:10: fatal error: offsets.h: No such file or directory
   26 | #include <offsets.h>
      |          ^~~~~~~~~~~
compilation terminated.
[8/369] Building C object drivers/display/CMakeFiles/..__..__..__Projects__Chips__test-origin-tft__drivers__display.dir/display_ili9xxx.c.obj
FAILED: drivers/display/CMakeFiles/..__..__..__Projects__Chips__test-origin-tft__drivers__display.dir/display_ili9xxx.c.obj 
C:\ncs\toolchains\c57af46cb7\opt\zephyr-sdk\arm-zephyr-eabi\bin\arm-zephyr-eabi-gcc.exe -DKERNEL -DLV_CONF_INCLUDE_SIMPLE=1 -DLV_CONF_PATH=C:/ncs/v2.5.1/zephyr/modules/lvgl/lv_conf.h -DNRF52840_XXAA -DPICOLIBC_INTEGER_PRINTF_SCANF -D_FORTIFY_SOURCE=1 -D_POSIX_C_SOURCE=200809 -D__LINUX_ERRNO_EXTENSIONS__ -D__PROGRAM_START -D__ZEPHYR__=1 -IC:/ncs/v2.5.1/zephyr/include -Izephyr/include/generated -IC:/ncs/v2.5.1/zephyr/soc/arm/nordic_nrf/nrf52 -IC:/ncs/v2.5.1/zephyr/lib/posix/getopt/. -IC:/ncs/v2.5.1/zephyr/soc/arm/nordic_nrf/common/. -IC:/ncs/v2.5.1/nrf/include -IC:/ncs/v2.5.1/nrf/tests/include -IC:/ncs/v2.5.1/modules/hal/cmsis/CMSIS/Core/Include -IC:/ncs/v2.5.1/zephyr/modules/cmsis/. -IC:/ncs/v2.5.1/modules/hal/nordic/nrfx -IC:/ncs/v2.5.1/modules/hal/nordic/nrfx/drivers/include -IC:/ncs/v2.5.1/modules/hal/nordic/nrfx/mdk -IC:/ncs/v2.5.1/zephyr/modules/hal_nordic/nrfx/. -IC:/ncs/v2.5.1/modules/lib/gui/lvgl/src -IC:/ncs/v2.5.1/zephyr/modules/lvgl/. -IC:/ncs/v2.5.1/modules/debug/segger/SEGGER -IC:/ncs/v2.5.1/modules/debug/segger/Config -isystem C:/ncs/v2.5.1/nrfxlib/crypto/nrf_cc310_platform/include -fno-strict-aliasing -Os -imacros C:/Projects/Chips/test-origin-tft/build/zephyr/include/generated/autoconf.h -fno-printf-return-value -fno-common -g -gdwarf-4 -fdiagnostics-color=always -mcpu=cortex-m4 -mthumb -mabi=aapcs -mfp16-format=ieee --sysroot=C:/ncs/toolchains/c57af46cb7/opt/zephyr-sdk/arm-zephyr-eabi/arm-zephyr-eabi -imacros C:/ncs/v2.5.1/zephyr/include/zephyr/toolchain/zephyr_stdint.h -Wall -Wformat -Wformat-security -Wno-format-zero-length -Wno-pointer-sign -Wpointer-arith -Wexpansion-to-defined -Wno-unused-but-set-variable -Werror=implicit-int -fno-pic -fno-pie -fno-asynchronous-unwind-tables -ftls-model=local-exec -fno-reorder-functions --param=min-pagesize=0 -fno-defer-pop -fmacro-prefix-map=C:/Projects/Chips/test-origin-tft=CMAKE_SOURCE_DIR -fmacro-prefix-map=C:/ncs/v2.5.1/zephyr=ZEPHYR_BASE -fmacro-prefix-map=C:/ncs/v2.5.1=WEST_TOPDIR -ffunction-sections -fdata-sections --specs=picolibc.specs -D_POSIX_THREADS -std=c99 -MD -MT drivers/display/CMakeFiles/..__..__..__Projects__Chips__test-origin-tft__drivers__display.dir/display_ili9xxx.c.obj -MF drivers\display\CMakeFiles\..__..__..__Projects__Chips__test-origin-tft__drivers__display.dir\display_ili9xxx.c.obj.d -o drivers/display/CMakeFiles/..__..__..__Projects__Chips__test-origin-tft__drivers__display.dir/display_ili9xxx.c.obj -c ../drivers/display/display_ili9xxx.c
In file included from C:/ncs/v2.5.1/zephyr/include/zephyr/app_memory/app_memdomain.h:9,
                 from C:/ncs/v2.5.1/zephyr/include/zephyr/rtio/rtio.h:31,
                 from C:/ncs/v2.5.1/zephyr/include/zephyr/drivers/spi.h:29,
                 from ../drivers/display/display_ili9xxx.h:13,
                 from ../drivers/display/display_ili9xxx.c:9:
C:/ncs/v2.5.1/zephyr/include/zephyr/linker/linker-defs.h:26:10: fatal error: offsets.h: No such file or directory
   26 | #include <offsets.h>
      |          ^~~~~~~~~~~
compilation terminated.
henrikbrixandersen commented 8 months ago

Thank you for reporting this. However - unless you are able to reproduce this issue with upstream Zephyr main - please report issues with the nRF Connect SDK (NCS) on the Nordic Semiconductor DevZone.

paultino commented 8 months ago

However - unless you are able to reproduce this issue with upstream Zephyr main - please report issues with the nRF Connect SDK (NCS)

Did you look carefully at my initial message? There is the main branch of Zephyr. NSC appeared due to research into the problem of not including drivers from the application folder when building the application. I asked at discord, they ignored me. I asked in Nordic, they answered. Successful building occurs only when these drivers are copied inside driver folder into main Zephyr. Today they repeated this problem (not enabling drivers) at their stand.

de-nordic commented 8 months ago

Eh... command line or Kconfig list to reproduce would be nice, huh?

I have tried building Hello World adding offsets.h to main and flash_shell adding the offsets.h to soc_nrf_flash.c. Both built for nrf52840dk_nrf52840 with the Zephyr at a9779eca400e67e2eebfc1f774eb4186efdc7a96. Both built with -DCONFIG_I2C=y, and looks ok. The example that is not building with sdk-nrf but can also be tried with the Zephyr: https://github.com/robertpaananen/mre_bug_offsets_h

west build -d builds/nrf -b nrf52840dk_nrf52840 ../ncs1/mre_bug_offsets_h/ -DCONFIG_I2C=y

fails, but when you re-try build it builds fine. Seems that the offsets.h gets generated after the file, triggering the problem, is compiled.

eriksage commented 7 months ago

I see the same issue on SDK 0.16.5-1, Zephyr v3.6.0 with board nucleo_f767zi. (Using SPI and CAN) However it is not consistent. Sometimes it builds from a clean build folder without issues, other times it complains about the missing . About 50/50. Building the second time always succeeds.

Because of this random behavior and that some report that it helps to add includes, could it be that this is caused by the parallel nature of ninja and that it sometimes manage to create offsets.h in time?

josuah commented 6 months ago

Would there be a missing dependency somewhere in some CMakeLists.txt? If the build is parallel, one file could be built before another, and depending on the order, there could be a missing dependency or not?

A manual workaround is to generate the offsets.h explicitly:

~/zephyrproject$ cmake --build build --target offsets_h
[1/1] Generating include/generated/offsets.h
~/zephyrproject$
htcrane commented 5 months ago

I am also seeing this with the following:

It occurs anytime you perform a pristine build or have otherwise nuked the build directory. If you build again after the failure (without the pristine flag), the process will continue as normal. I believe offsets.h is generated by the Zephyr SDK, and therefore is not built early enough in the compiling process. Hence, the failure upon first clean builds but success after continuing the failed build.

Looking a the offsets.h file (in your build dir/zephyr/include/generated), there are no external includes so ideally, it should be one of the first built with a gated build mechanism (@eriksage, I agree it is the parallelism causing issues).

henrikbrixandersen commented 4 months ago

It occurs anytime you perform a pristine build or have otherwise nuked the build directory.

Please provide concise steps for how to reproduce this (list the exact commands used along with full error output). I am not able to reproduce this just by doing a pristine build.

alexanderwachter commented 4 months ago

I see the same issue (lates toolchain 0.16.6, running on linux, zephyr 3.6.0) building an off-tree test written in C++. It is very rare and rebuilding helps always. However, it is annoying in the CI.

Building:

./zephyr/scripts/twister --inline-logs -v -p nucleo_g474re --fixture A0_A1_bridged --fixture UART_loopback -O build_nucleo_samples_tests -T LibMCU/samples -T LibMCU/tests

Error:

ccache /home/zdev/zephyr-sdk-0.16.6/arm-zephyr-eabi/bin/arm-zephyr-eabi-g++ -DCORE_CM4 -DHSE_VALUE=24000000 -DKERNEL -DK_HEAP_MEM_POOL_SIZE=0 -DPICOLIBC_LONG_LONG_PRINTF_SCANF -DSTM32G474xx -DTC_RUNID=fdf47232876ed736225ffbe0374d854d -DUSE_FULL_LL_DRIVER -DUSE_HAL_DRIVER -DLINUX_ERRNO_EXTENSIONS -D__PROGRAM_START -DZEPHYR=1 -I/home/zdev/LibMCU/LibMCU/include -I/home/zdev/LibMCU/modules/hal/cmsis/CMSIS/Core/Include -I/home/zdev/LibMCU/zephyr/modules/cmsis/. -I/home/zdev/LibMCU/modules/hal/stm32/stm32cube/stm32g4xx/soc -I/home/zdev/LibMCU/modules/hal/stm32/stm32cube/stm32g4xx/drivers/include -I/home/zdev/LibMCU/modules/hal/stm32/stm32cube/stm32g4xx/drivers/include/Legacy -I/home/zdev/LibMCU/modules/hal/stm32/stm32cube/common_ll/include -I/home/zdev/LibMCU/modules/EmbeddedLib/include -isystem /home/zdev/LibMCU/zephyr/include -isystem /home/zdev/LibMCU/build_nucleo_samples_tests/nucleo_g474re/LP5569/libmcu.drivers.lp5569/zephyr/include/generated -isystem /home/zdev/LibMCU/zephyr/soc/arm/st_stm32/stm32g4 -isystem /home/zdev/LibMCU/zephyr/lib/libc/common/include -isystem /home/zdev/LibMCU/zephyr/drivers -isystem /home/zdev/LibMCU/zephyr/soc/arm/st_stm32/common -isystem /home/zdev/LibMCU/zephyr/subsys/testsuite/include -isystem /home/zdev/LibMCU/zephyr/subsys/testsuite/ztest/include -O3 -DNDEBUG -fno-strict-aliasing -Os -fcheck-new -std=c++20 -Wno-register -Wno-volatile -fno-exceptions -fno-rtti -imacros /home/zdev/LibMCU/build_nucleo_samples_tests/nucleo_g474re/LP5569/libmcu.drivers.lp5569/zephyr/include/generated/autoconf.h -fno-common -g -gdwarf-4 -fdiagnostics-color=always -mcpu=cortex-m4 -mthumb -mabi=aapcs -mfp16-format=ieee -mtp=soft --sysroot=/home/zdev/zephyr-sdk-0.16.6/arm-zephyr-eabi/arm-zephyr-eabi -imacros /home/zdev/LibMCU/zephyr/include/zephyr/toolchain/zephyr_stdint.h -Wall -Wformat -Wformat-security -Wno-format-zero-length -Wdouble-promotion -Wpointer-arith -Wexpansion-to-defined -Wno-unused-but-set-variable -fno-pic -fno-pie -fno-asynchronous-unwind-tables -ftls-model=local-exec -fno-reorder-functions --param=min-pagesize=0 -fno-defer-pop -fmacro-prefix-map=/home/zdev/LibMCU/LibMCU/tests/LP5569=CMAKE_SOURCE_DIR -fmacro-prefix-map=/home/zdev/LibMCU/zephyr=ZEPHYR_BASE -fmacro-prefix-map=/home/zdev/LibMCU=WEST_TOPDIR -ffunction-sections -fdata-sections --specs=picolibc.specs -Wextra -Wshadow -Wnon-virtual-dtor -Wold-style-cast -Wcast-align -Wunused -Woverloaded-virtual -Wpedantic -Wconversion -Wsign-conversion -Wnull-dereference -Wformat=2 -Wno-error=unknown-pragmas -Wno-unknown-pragmas -Weffc++ -Werror -Wmisleading-indentation -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wuseless-cast -pedantic -MD -MT modules/LibMCU/src/CMakeFiles/gmat-lib-mcu.dir/LP5569/LP5569.cpp.obj -MF modules/LibMCU/src/CMakeFiles/gmat-lib-mcu.dir/LP5569/LP5569.cpp.obj.d -o modules/LibMCU/src/CMakeFiles/gmat-lib-mcu.dir/LP5569/LP5569.cpp.obj -c /home/zdev/LibMCU/LibMCU/src/LP5569/LP5569.cpp In file included from /home/zdev/LibMCU/zephyr/include/zephyr/app_memory/app_memdomain.h:9, from /home/zdev/LibMCU/zephyr/include/zephyr/rtio/rtio.h:31, from /home/zdev/LibMCU/zephyr/include/zephyr/drivers/i2c.h:28, from /home/zdev/LibMCU/LibMCU/src/LP5569/LP5569.cpp:4: /home/zdev/LibMCU/zephyr/include/zephyr/linker/linker-defs.h:26:10: fatal error: offsets.h: No such file or directory 26 | #include

hraftery commented 4 months ago

I looked into this because it's been causing us strife for quite a while. The build process is confounding enough without worrying about intermittent tooling issues like this.

I found that according to the build system docs the offsets.h file is created during pre-build. Critically that comes after the "Configuration Phase" as the first step during the "Build Phase". So this is under the control of either make or ninja.

In my case it is ninja, so I tracked down my build.ninja "makefile". Of course, it is 4.3MB and ninja run builds without inter-dependencies in parallel, so what could possibly go wrong?

I ran ninja -t deps and ninja -t graph to get a view into the dependency tree. The relevant build target was easy to spot in the graph:

Screenshot 2024-05-15 at 2 13 10 PM

So to understand why it's not getting prioritised we simply need to look at it in context within the overall build. I've drawn a big red box around the previous screenshot to make it easy to spot.

Screenshot 2024-05-15 at 3 08 52 PM

Yeah. So I'm still looking into it...

hraftery commented 4 months ago

Oh man, that was easy.

Everything that was included using target_sources had dependencies like zephyr_generated_headers. So did drivers I'd added using add_subdirectory, but which have a path already known to Zephyr, like input drivers. They get a line in the build.ninja file like:

cmake_object_order_depends_target_drivers__input: phony || [...] zephyr/zephyr_generated_headers

However, drivers in an unknown path, which are included in my CMakeLists.txt file using a combination of add_subdirectory and target_link_library, get no such dependency. I found an old thread which got me on the right track.

All I had to do, was add add_dependencies(LIBRARY_NAME, offsets_h) to my CMakeLists.txt file, and clean builds are now fast and successful.

If you're following the Zephyr library naming convention, as I am, that would be something like add_dependencies(drivers__fuel_gauge__max17043 offsets_h). But if you're relying on auto-generated names (which use a path relative to the ZEPHYR_BASE), like maybe the OP is, it might be something like add_dependencies(..__..__..__Projects__Chips__test-origin-tft__drivers__display offsets_h).

I'm not sure why this dependency isn't included transitively already...

henrikbrixandersen commented 4 months ago

CC: @tejlmand

sidharth-prabukumar commented 4 months ago

@hraftery Using zephyr_library_add_dependencies(offsets_h) worked for me

github-actions[bot] commented 3 weeks ago

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.