Out-of-tree custom board example missing from documentation

w8jcik commented 3 years ago

After spending considerably long time trying to use CONFIG_ARCH_BOARD_CUSTOM I was still not able to.

There is either a bug or it is difficult to use.

Setup

With following directory structure

/base
    /nuttx (incubator-nuttx)
    /apps (incubator-nuttx-apps)
/board (empty folder)

cd base/nuttx ; ./tools/configure.sh hymini-stm32v:nsh2

Builds and runs.

Then I am trying to use CONFIG_ARCH_BOARD_CUSTOM functionality, with hymini-stm32v:nsh2, by making a copy of it and trying to build NuttX with the copy. This particular board is just an example.

First attempt

Copied content of https://github.com/apache/incubator-nuttx/tree/master/boards/arm/stm32/hymini-stm32v into /board.

Using make menuconfig selected

CONFIG_ARCH_BOARD_CUSTOM=y
CONFIG_ARCH_BOARD_CUSTOM_NAME=custom-board  # description says it is not relevant
CONFIG_ARCH_BOARD_CUSTOM_DIR=../../board
CONFIG_ARCH_BOARD_CUSTOM_DIR_RELPATH=y

Issues at this step: 1) If I specify CONFIG_ARCH_BOARD_CUSTOM_DIR wrongly, it is not possible to run make menuconfig anymore. 2) If I specify correctly, it is not building.

make[2]: Entering directory '/home/user/dev/project/board/src'
make[2]: *** No rule to make target 'libboard.a'. Stop.
make[2]: Leaving directory '/home/user/dev/project/board/src'
make[1]: *** [Makefile:168: board/libboard.a] Error 2
make[1]: Leaving directory '/home/user/dev/project/base/nuttx/arch/arm/src'
make: *** [tools/Makefile.unix:423: nuttx] Error 2

Second attempt

I guess tools/configure.sh should be used for this somehow

tools/configure.sh -h  # doesn't give any related help

USAGE: tools/configure.sh [-E] [-e] [-l|m|c|g|n] [L] [-a <app-dir>] <board-name>:<config-name> [make-opts]

Where:
  -E enforces distclean if already configured.
  -e performs distclean if configuration changed.
  -l selects the Linux (l) host environment.
  -m selects the macOS (m) host environment.
  -c selects the Windows host and Cygwin (c) environment.
  -g selects the Windows host and MinGW/MSYS environment.
  -n selects the Windows host and Windows native (n) environment.
  Default: Use host setup in the defconfig file
  Default Windows: Cygwin
  -L  Lists all available configurations.
  -a <app-dir> is the path to the apps/ directory, relative to the nuttx
     directory
  <board-name> is the name of the board in the boards directory
  configs/<config-name> is the name of the board configuration sub-directory
  make-opts directly pass to make

Analysing mailing lists and the script I found that relative path can be used as parameter, but not sure how exactly, so a bit of guessing is involved

tools/configure.sh ../../board
File Make.defs could not be found

tools/configure.sh ../../board:nsh2
Directory for ../../board:nsh2 does not exist.

# This runs, but it is not consistent with normal usage of this command and absent from help or documentation
tools/configure.sh ../../board/configs/nsh2

It builds, but it just configures hymini-stm32v:nsh2 as far as I can tell.

So I am changing defconfig in the copy (/board) before running configure.sh again

/board/configs/nsh2/defconfig

-CONFIG_ARCH_BOARD="hymini-stm32v"
-CONFIG_ARCH_BOARD_HYMINI_STM32V=y

+CONFIG_ARCH_BOARD_CUSTOM=y
+CONFIG_ARCH_BOARD_CUSTOM_NAME="custom-board"
+CONFIG_ARCH_BOARD_CUSTOM_DIR="../../board"
+CONFIG_ARCH_BOARD_CUSTOM_DIR_RELPATH=y

make distclean
tools/configure.sh ../../board/configs/nsh2

These configuration variables can be found in the mailing list, this is why I am trying to use them. But the project is not building afterwards.

make[2]: Nothing to be done for 'clean_context'.
make[2]: Leaving directory '/home/user/dev/project/base/nuttx/arch/arm/src'
make[1]: Leaving directory '/home/user/dev/project/base/nuttx/tools'
make[1]: Leaving directory '/home/user/dev/project/base/nuttx'
Create version.h
LN: include/arch to arch/arm/include
LN: arch/arm/src/board to /home/user/dev/project/base/nuttx//src
LN: arch/arm/src/chip to arch/arm/src/stm32
LN: /home/user/dev/project/base/nuttx/drivers/platform to /home/user/dev/project/base/nuttx/drivers/dummy
No directory at /home/user/dev/project/base/nuttx//src
make: *** [tools/Makefile.unix:293: arch/arm/src/board] Error 1

Conclusion

This functionality is difficult to use.
A receipe is missing from documentation.

I am not sure what I am missing, analysed Makefiles a bit and searched through different mailing lists of NuttX trying some solutions present there, but it doesn't help me solve this. It would be great if it would be easier. Recipe in the documentation would help a lot.

Reading this doesn't help much

Articles above provide information about links that build system creates, but these links seem to be fine, pointing to external /board directory.

If you are using this logic, could you provide a short example how it should be used?

Then I think it should be included in documentation.

Appendix

$ tools/configure.sh -e ../../board/configs/nsh2 | grep -E "^LN:"
LN: include/arch to arch/arm/include
LN: include/arch/board to /home/user/dev/project/base/nuttx/../../board/include
LN: include/arch/chip to arch/arm/include/stm32
LN: arch/arm/src/board to /home/user/dev/project/base/nuttx/../../board/src
LN: arch/arm/src/chip to arch/arm/src/stm32
LN: /home/user/dev/project/base/nuttx/drivers/platform to /home/user/dev/project/base/nuttx/drivers/dummy
LN: platform/board to /home/user/dev/project/base/apps/platform/dummy

Using master branch.

Ouss4 commented 3 years ago

The correct way to use it is: ./tools/configure.sh ../path_to_board/configs/config_name. In config_name directory you should have both a defconfig and a Make.defs files (Although, a recent PR made it possible to have a shared Make.defs in the scripts directory similar to in-tree boards.) Note that this isn't really "different" from in-tree boards. In-tree boards support two notations: the board/defconfig and the board:defconfig. Custom boards support only the board/defconfig.

Please, make sure you are setting the paths correctly: "No directory at /home/user/dev/project/base/nuttx//src" In the past, when I was seeing similar issues to this one, it was always because of a bad path. For a reference here are the relevant CONFIGs_:

CONFIG_ARCH_BOARD_CUSTOM=y
CONFIG_ARCH_BOARD_CUSTOM_DIR="../boards/board-name"
CONFIG_ARCH_BOARD_CUSTOM_DIR_RELPATH=y
CONFIG_ARCH_BOARD_CUSTOM_NAME="board-name"

PS.: I just built an old out-of-tree board that I haven't touched in a while.

Regarding documentation, I was always willing to do that but always get discouraged by the chicken and egg problem of generating the defconfig or hand editing it.

Ouss4 commented 3 years ago

I also notice that you copied a board that uses the custom and drivers directories. You need those too.

protobits commented 3 years ago

Thank you for opening this issue, I'm always annoyed by the same issue but since I don't do this really often I neglected this.

Regarding documentation, I was always willing to do that but always get discouraged by the chicken and egg problem of generating the defconfig or hand editing it.

That's exactly the problem: when you create a custom board you need an existing one as template (writing everything from scratch does not make sense). If you copy a non-custom board and try to configure it with configure.sh this immediately fails because the ARCH_BOARD_CUSTOM is not yet set. Hand-editing the config is not really something that should be normal for this.

What could we do to improve this? Of course we could make a "make-custom.sh" script but sounds a hack. Maybe we can add the ability to recognize an out-of-tree board to configure.sh and set the appropriate configs?

w8jcik commented 3 years ago

@Ouss4 regarding the issue of mine:

Please, make sure you are setting the paths correctly: "No directory at /home/user/dev/project/base/nuttx//src"

I didn't put quotes over path in defconfig, apparently they are required by Kconfig. But then it was stopping on this:

make[1]: Entering directory '/home/user/dev/project/base/nuttx/arch/arm/src'
make[2]: Entering directory '/home/user/dev/project/board/src'
make[2]: *** No rule to make target 'libboard.a'. Stop.
make[2]: Leaving directory '/home/user/dev/project/board/src'
make[1]: *** [Makefile:168: board/libboard.a] Error 2
make[1]: Leaving directory '/home/user/dev/project/base/nuttx/arch/arm/src'
make: *** [tools/Makefile.unix:425: nuttx] Error 2

I also notice that you copied a board that uses the custom and drivers directories. You need those too.

You mean common right? Yes this was missing, thank you a lot for help :D

common is not optional, Makefile needs to be there for build system to work. It is far from obvious.

@v01d Regarding documentation and improvement.

I think describing how to do it by copying an existing board is already great. Example don't need to be generic. Also I think this is typical workflow, you start with some similar development kit and change it in few places to make it run on your board. I can describe what I did to make a custom copy, based on this particular board and you might review it.

we could make a "make-custom.sh"

This is what web frameworks do, they have scripts to generate code based on templates. I think it would be perfectly fine to have such script which takes any board code, performs simple refactoring to avoid name collisions and gives you something which already runs. I will prepare it.

protobits commented 3 years ago

Note that I would prefer configure.sh to deal with this. If you are configuring an out-of-tree board it does not make any sense to interpret the configuration as an in-tree board. The 'make-custom.sh' is something that would eventually break IMHO.

w8jcik commented 3 years ago

If you are configuring an out-of-tree board it does not make any sense to interpret the configuration as an in-tree board.

I am not sure if I understand. Is there some other logic that handle out-of-tree boards? For simplicity out-of-tree and in-tree could be handled in the same way. I am just talking about a tool that clones board configurations in automated way, so people don't have to do it.

Maybe the issue here is that they are handled differently right now.

protobits commented 3 years ago

Well the configure script already needs to know if it is an out of tree board or not, since it will have to setup things differently. This is fact why using an ouf-of-tree board using an in-tree config fails. I believe it is mostly when trying to create links to board directory, which is driven by the config values. Ideally the configure script would do the right thing with respect to not breaking with such an input config.

On the other hand, there's of course some editing to be done for converting an in-tree board to an out-of-tree board: many macro names will change (depending on the board name), header files (and includes to them), Kconfig and Makefile depending on ARCH_BOARD_CUSTOM instead of the board-dependant symbol. All of this could indeed by made by a script, but I would generalize this to a "new-board.sh" script (since copying an in-tree board to create another in-tree board is also relatively common).

Of course if you write that you can also deal with modifying the configuration appropiately. But besides that, I feel that configure.sh breaking this way is not desireable either. Anyway, if you prefer to create this script and solve both tasks, I think it is not such a bad idea.

protobits commented 3 years ago

Note that a similar script could be useful for applications. In this case it is not necessary to supporting copying any existing application, but a specific one can be used (I'm not sure if there's a skeleton app of some sort, maybe "hello" app could be used for that instead). That would also be a nice addition.

Ouss4 commented 3 years ago

You mean common right? Yes this was missing, thank you a lot for help :D

Sorry, yes I meant common. Is everything building now?

Note that I would prefer configure.sh to deal with this.

I'm also not sure I understand this. configure.sh already deals with them, provided that everything is set correctly. From the user's perspective, in-tree and out-of-tree boards are, for the most part, handled the same way. I think what we agree on is that some config parts aren't obvious and lack documentation.

On the other hand, there's of course some editing to be done for converting an in-tree board to an out-of-tree board: many macro names will change (depending on the board name), header files (and includes to them), Kconfig and Makefile depending on ARCH_BOARD_CUSTOM instead of the board-dependant symbol.

Note, that most of that has been refactored in tools/Config.mk.

All of this could indeed by made by a script, but I would generalize this to a "new-board.sh" script (since copying an in-tree board to create another in-tree board is also relatively common).

FYI, there is this: https://github.com/apache/incubator-nuttx/blob/master/tools/initialconfig.c, however, I failed to make it work last time I tried.

protobits commented 3 years ago

Note, that most of that has been refactored in tools/Config.mk.

Ah, that is nice. How do things change now for custom boards?

All of this could indeed by made by a script, but I would generalize this to a "new-board.sh" script (since copying an in-tree board to create another in-tree board is also relatively common).

FYI, there is this: https://github.com/apache/incubator-nuttx/blob/master/tools/initialconfig.c, however, I failed to make it work last time I tried.

That is really useful, didn't know it existed (tools directory really needs some organization).

Ouss4 commented 3 years ago

Ah, that is nice. How do things change now for custom boards?

Before that refactoring, there were some hard-coded links in Make.defs files. Those were also another trap when copying boards from the tree. Example: https://github.com/apache/incubator-nuttx/blob/23952584866e754163ba384f6b6b71f123f37a8d/tools/Config.mk#L156-L165

Now, when using BOARD_DIR, the same Make.defs can be used for both in-tree and ouf-of-tree boards.

Ouss4 commented 3 years ago

Note that a similar script could be useful for applications. In this case it is not necessary to supporting copying any existing application, but a specific one can be used (I'm not sure if there's a skeleton app of some sort, maybe "hello" app could be used for that instead). That would also be a nice addition.

This would be easy, however. Copying an application (hello for instance) is just replacing the names and simlinking it to externals/. But yes, a script to do so could come in handy.

f4grx commented 3 years ago

Hello,

This is still not working OK.

copying a board config to a custom dir.

using ./configure.sh builds the original built-in board

changing the config file to point into the custom board fails to build libboard.a

The old mechanism where you could just ln your custom board in the nuttx tree worked better.

f4grx commented 3 years ago

make[1]: Entering directory '/home/slo/nut/nuttx/fs'
make[1]: 'libfs.a' is up to date.
make[1]: Leaving directory '/home/slo/nut/nuttx/fs'
make[1]: Entering directory '/home/slo/nut/nuttx/binfmt'
make[1]: 'libbinfmt.a' is up to date.
make[1]: Leaving directory '/home/slo/nut/nuttx/binfmt'
make[1]: Entering directory '/home/slo/nut/nuttx/arch/arm/src'
make[2]: Entering directory '/home/slo/nut/my_own_nucleo/src'
echo "damn you libboard"
damn you libboard
make[2]: Leaving directory '/home/slo/nut/my_own_nucleo/src'
LD: nuttx
arm-none-eabi-ld: cannot find -lboard
make[1]: *** [Makefile:156: nuttx] Error 1
make[1]: Leaving directory '/home/slo/nut/nuttx/arch/arm/src'
make: *** [tools/Makefile.unix:422: nuttx] Error 2
$ cat /home/slo/nut/my_own_nucleo/src/Makefile
context:
        echo "damn you context"
libboard.a:
        echo "damn you libboard"

The issue seems to be a missing Makefile in the src subdirectory of the custom board. If I add one with a small echo comment in it, I see it displayed.

the makefile is probably optional for built-in boards but it is required for custom boards.

So WHAT makefile has to be there and with what rules?

edit: The makefile in stm32/common/src does not work when copied in my_board/src

f4grx commented 3 years ago

OK so the solution to this was given by Abdelatif Guettouche on the mailing list

I just copied the nucleo-fe446re board without the common directory.

Here are the steps I took:
Add the correct custom board CONFIGs to nsh/defconfig (and remove some
of the old ones)
Rename Make.defs to Makefile
Add include $(TOPDIR)/boards/Board.mk to the end of the file.

./tools/configure.sh ../boards/nucleo-f446re/configs/nsh
make

Builds fine.

The trick about RENAMING a make.defs to Makefile is probably not normal. This process has to be documented somewhere in the official doc under "how to make an external board config"

trns1997 commented 8 months ago

@f4grx just a note we tried out-of-tree build for our custom board and it worked without the need of renaming Make.defs to Makefile and without the need to add include $(TOPDIR)/boards/Board.mk. We simply added the following options in our defconfig:

# CONFIG_ARCH_BOARD_CUSTOM_DIR_RELPATH is not set
CONFIG_ARCH_BOARD_CUSTOM=y
CONFIG_ARCH_BOARD_CUSTOM_DIR="YOUR BOARD DIR"
CONFIG_ARCH_BOARD_CUSTOM_NAME="myCustomBoard"

Note the chip we tested with the xmc4700.

This worked:

./tools/configure.sh ABSOLUTE_PATH_TO_CUSTOM_BOARD/configs/myCustomBoardConfig
make -j8

apache / nuttx