Simulator export fails

[nicolas71640]

I am trying to build my own app using the export target with the simulator. And I've run into few issues : To be clear, here's the procedure I following :

1. ./tools/configure.sh sim:nsh
2. make export
3. tar -xf nuttx-export-12.4.0.tar.gz
4. My CMakeList.txt file :

   cmake_minimum_required(VERSION 3.16)
   project(test)
   include(../nuttx-export-12.4.0/scripts/toolchain.cmake)
   add_executable(test main.c)

5. cmake . && make and here's the output ld: unrecognized option '-Wl,--gc-sections'

Indeed in the target.cmake of the export the LDFLAGS are : "-Wl,--gc-sections -Wl,-Ttext-segment=0x400000 -Wl,-Map=/home/nuttxspace/nuttx/nuttx.map")

Which can not work because we passing the -Wl argument directly to the linker... First issue.

6. To go further, I have just modified the target.cmake, removing the -Wl before the arguments. Then the linker returns this error : ld: read in flex scanner failed With some research, this error is a the way of the linker to say that it can't read some arguments. Some tests showed me that the -T{LINKER_SCRIPT} argument (toolchain.cmake) was the culprit. Here's the line where we set the LINKER_SCRIPT variable : set(LINKER_SCRIPT ${NUTTX_PATH}/scripts/${LDNAME}) Problem, in the target.cmake the ${LDNAME} is empty: set(LDNAME "") The LDNAME variable is set in the Export.mk file :

   ifneq ($(ARCHSCRIPT),)
   LDPATH = $(call CONVERT_PATH,$(ARCHSCRIPT))
   LDNAME = ${notdir ${LDPATH}}
   LDDIR = ${dir ${LDPATH}}
   endif

For now, I've stopped there the investigation, but I will go further in this. These errors make me think that the simulator export is bugged. What do you think ? Is it possible that it isn't really used and it is now outdated ? Or is there something I'm doing wrong ? By the way, without the export, it works perfectly well (tested with nsh). ping @leducp

[acassis]

Hi @nicolas71640 do you need to use CMakefile? Maybe the export building using Makefile still working. There is an old Documentation (that didn't migrate to our new Documentation/) here: https://cwiki.apache.org/confluence/display/NUTTX/Building+NuttX+with+Applications+Outside+of+the+Source+Tree

[acassis]

@xuxin930 since you are fixing the CMake, could you please take a look?

[xuxin930]

hi @acassis @nicolas71640 Thank you for your reminder 😊 The current CMake base does not support the import export feat. We considered using cmake installduring the design, but it is not perfect yet.

You can first consider using the Makefile base compilation method.

I will mark an action and try to add this feature of CMake as soon as possible.

[leducp]

I think there is a quiproquo here: we are using CMake as our project tool, but we build NuttX with Makefile. The exported CMake toolchain file is based on the export.sh script and I imagine that the variables are coming from the Makefile mechanism since it is the one we are using to build NuttX.

My understanding is that NuttX Makefile or CMake should do the same things (maybe it is buggy) but at the end, the NuttX export archive consumption is independent of the NuttX procedure to build it, using Make or CMake to do so

[nicolas71640]

Thank you all for your help. As I understand it, @acassis and @xuxin930 you suggest to use the export AND Import script to us the Makefile to build the final binary. I actually didn't know that an import.sh and import target existed... I've tried this morning. Works perfectly with a random arch/board, but doesn't work with the simulator. First, same issue with the linker flag (-Wl,--gc-section instead of --gcc-section in the LDFLAGS), then this issue : ld: cannot open linker script file -o: No such file or directory

Am I doing something wrong ?

Just followed a readme that I've found in the source :

    $ cd nuttx
    $ ./tools/configure.sh sim:nsh
    $ make V=1 -j7
    $ make export V=1
    $ cd ../apps
    $ ./tools/mkimport.sh -z -x ../nuttx/nuttx-export-*.tar.gz
    $ make import V=1

[acassis]

@nicolas71640 I got same errors you got:

LD: nuttx
ld --entry=__start -Wl,--gc-sections -Wl,-Ttext-segment=0x400000 -Wl,-Map=/home/alan/nuttxspace/nuttx/nuttx.map -L/home/alan/nuttxspace/apps/import/libs -L "/usr/lib/gcc/x86_64-linux-gnu/11"  \
  -L/home/alan/nuttxspace/apps/import/scripts -T \
  -o nuttx /home/alan/nuttxspace/apps/import/startup/sim_head.o /home/alan/nuttxspace/apps/import/startup/sim_hostfs.o /home/alan/nuttxspace/apps/import/startup/sim_hostirq.o /home/alan/nuttxspace/apps/import/startup/sim_hostmemory.o /home/alan/nuttxspace/apps/import/startup/sim_hostmisc.o /home/alan/nuttxspace/apps/import/startup/sim_hosttime.o /home/alan/nuttxspace/apps/import/startup/sim_hostuart.o /home/alan/nuttxspace/apps/builtin/builtin_list.c.home.alan.nuttxspace.apps.builtin_1.o /home/alan/nuttxspace/apps/builtin/builtin_list.c.home.alan.nuttxspace.apps.builtin.o /home/alan/nuttxspace/apps/builtin/exec_builtin.c.home.alan.nuttxspace.apps.builtin_1.o /home/alan/nuttxspace/apps/builtin/exec_builtin.c.home.alan.nuttxspace.apps.builtin.o  --start-group \
  /home/alan/nuttxspace/apps/libapps.a -lsched -ldrivers -lboards -lc -lmm -larch -lapps -lfs -lbinfmt -lboard  -lgcc  --end-group
ld: unrecognized option '-Wl,--gc-sections'
ld: use the --help option for usage information
make[1]: *** [Makefile:124: .import] Error 1
make[1]: Leaving directory '/home/alan/nuttxspace/apps'
make: *** [Makefile:142: import] Error 2

Probably it is happening because normally people use it mainly with real boards to integrating NuttX kernel as library and avoid using our building system in production.

@xiaoxiang781216 did you or your team use make export/import with simulator?

[xiaoxiang781216]

Yes, but only for Makefile

[nicolas71640]

@xiaoxiang781216 What do you mean by "only for Makefile" ? as said I said here : https://github.com/apache/nuttx/issues/11863#issuecomment-1991042732 without any cmake help, the export/import thing fails... Am I misunderstanding something here ?

[xiaoxiang781216]

we export and import only with make, not try export with make and build with cmake. Your usage doesn't coverage by daily ci or even mention in document.

[nicolas71640]

I'm not using cmake. I'm using make targets "export" and "import".

[nicolas71640]

Hi everyone ! After a bit of investigation, I'm beginning to understand how the simulator work, and therefore how it compiled/linked. If you allow me, I'll sum up what I've understood and I haven't...

As the simulator runs on the host, it must replace some of its symbols by nuttx symbol. For instance, the method puts, shouldn't print something on the output but use the "mocked" serial console of nuttx (passing through the UART for instance). This map of symbols is defined in nuttx-names.in.

So the first step of the linking, is to make a "partially linked" lib (nuttx.rel) containing nuttx code, replacing the libc symbols by its own. Then the second step is to take this nuttx.rel lib and link it to the interface with the host (all the HOSTOBJS). An other object is linked in this final binary is the sim_head.o, containing the entry point of the simulator.

Did I understand correctly what's happening here ?

Now, for the export. My first though was that we needed this nuttx.rel to be linked on the final target. So put that nuttx.rel in the archive. Apparently this is not the strategy. I see that we export the nuttx-names.dat in the archive (in libs/ ). Why not, but I can't find where this file is used to replace the symbols in the import scripts... An idea ?

Therefore, I have absolutely no idea how the simulator export could work...

[acassis]

@nicolas71640 I think your assumptions/findings are mostly corrects.

@xiaoxiang781216 @anchao any idea how to make the simulator to export the symbols correctly?

[xiaoxiang781216]

@nicolas71640 I think your assumptions/findings are mostly corrects.

Yes, sim has verry special linking process, which isn't fit well with the current import/export

@xiaoxiang781216 @anchao any idea how to make the simulator to export the symbols correctly?

We didn't try export/import sim before because:

1. Simulator doesn't support protected and kernel mode
2. All sim arch specific code is open source

Both make the usage of sim export/import is very seldom. @nicolas71640 which functionality drive you want this feature?

[nicolas71640]

We wanted to use the simulator for test purposes. We thought it would be a simple way to have our application packaged with nuttx in a very testable/instrumentable environment as linux. We are also planning to have tests on the board, but these tests would be slower and less instrumentable, so we wanted to have both.

The other option was a qemu obviously, but we thought the simulator would be lighter and faster, and maybe simpler to set up in our CI. Now, if you have advices, we are willing to rethink our strategy.

Exporting the nuttx.rel (and nuttx.ld), I have actually succeed to link the final binary, BUT not only with ld, but passing via gcc (as we do it the sim Makefile). I assume it uses standard libs that I don't put with ld (but I can't figure out which one I'm missing with ld). So I f I want to make the export/import stuff works for the sim, I'll have to add some var, and a specific strategy only for the simulator...

[xiaoxiang781216]

Qemu may better than sim in this case, because both toolchain and export/import is same as the real device. Most arch (Cortex-A/M/R, ARM64, RISCV32/64 and XTENSA) has the demo board, so NuttX qemu can be setup quickly. BTW, since toolchain/arch is same as your final product, the library built with exported SDK can be used with your hardware directly, which is impossible for simulator.

[nicolas71640]

All right ! Well, thank you @xiaoxiang781216. I think we are going to follow your advice ! Closing this issue then ;)