xiaoxiang781216
commented
4 years ago So, you plan to generate the source code from device tree? The device tree concept disappear after the tool finish the conversion. This approach is totally different from Linux which I am working with more than five years. Did you studied how device tree is used in other OS(e.g. Linux or Zephyr) and summary pros and cons for the different approach? I am working with Linux kernel more than ten years, the most important design decision for Linux driver from my view is just two things: 1.Introduce the common driver model from 2000~: https://www.kernel.org/doc/html/latest/driver-api/driver-model/index.html 2.Inroduce the device tree from 2010~: https://github.com/torvalds/linux/blob/master/Documentation/devicetree/usage-model.txt https://github.com/torvalds/linux/blob/master/Documentation/devicetree/booting-without-of.txt And give me a strong impresson how Linux combine these two piece concept in a flexiable way and fix many driver problems I found ten years ago with a a elegant solution. So I would sugest to study how other OS(at least Linux and Zephy) use the device tree before we start porting the device tree to NuttX.
protobits
patacongo
justinmclean
btashton
davids5
jerpelea
cederom
bcb78
I've been thinking and researching for some time ways to improve how boards are supported in NuttX. I would like to describe a mechanism I believe would be useful in order to get feedback from maintainers. I would be willing to work on this if there is interest. PS: sorry if this is lengthy but I believe it is better to be detailed about it to the idea and rationale across
Problem Statement
Currently NuttX supports a large number of boards, which involve many different architectures and MCU families. To support a board it is necessary to configure NuttX for the corresponding MCU according to its on-chip resources and on-board devices (anything soldered to the PCB). Moreover, board logic includes the instantiation of off-board devices that may or may not be used (eg. external sensors connected via pin headers, an arduino shield, etc).
In general, much of the board logic involves: 1) resource-mapping: clock, pin, timer, etc. setting selection via defines. Device driver initialization also involves resource-mapping, for example passing the chosen communication bus (I2C1) where a given device instance is found 2) Glue code between upper-level device world and low-level architecture world: for example, to configure a given pin as interrupt source and register a given interrupt service routine using architecture-dependant functions (eg: stm32_*) in order to callback into the upper-level of the driver.
While going over various board directories it is easy to find logic glue-code duplication, which varies in complexity from a simple function call passing one argument (eg, the timer number, the I2C port, etc) to more complex code involving callbacks as mentioned above. This duplication has led to divergence in implementations (ie. one version of the duplicate has been worked on while not the other, code is reorganized, etc). By looking at this code it is evident that there's no real dependence on a given particular board but only to the architecture/sub-architecture since the API used to implement this code is the same for the whole architecture/sub-architecture (eg, stm32_i2cbus_initialize). At most, the difference is which particular chip or board resource instance is chosen (eg, according to available I2C ports of the chip and for each port, the pins selected for this function).
This similarity motivates the move of all this code to a common location (such as boards/arm/stm32/common, as I started as part of #1006) which is the first step to improve maintainability. However, once moved to "board-agnostic" location, these "resource identifiers" (port number, timer number, pin configuration) needs to be supplied as parameter, since when this logic resides in a board-specific directory these information is hardcoded via macro definitions. An example of this in #1006 is: header: https://github.com/apache/incubator-nuttx/pull/1006/commits/44972b32ab5dc3c716855fc9f0c4d7781df510e2#diff-42be725a6a20b5b7f6fd1bffc4b96807R38 source: https://github.com/apache/incubator-nuttx/pull/1006/commits/44972b32ab5dc3c716855fc9f0c4d7781df510e2#diff-dc7f3d864bc967d588bca1dd29e65bb8R147
This solution is not ideal since now there is a global variable occupying space for each case for when this happens. An reasonable alternative would be to again use hardcoded values, ideally generated from Kconfig entries. In fact, all clock logic selection usually performed manually in-code via defines should also be selectable via Kconfig. The limitation of this approach is that for pin definitions it is not possible nor reasonable to directly generate a macro entry which contains all necessary bits.
Going back to the reasons of duplicate logic in boards is the initialization of off-board devices. Many boards now include configurations and conditionally-compiled initialization calls for plausible testing scenarios involving devices that the user may potentially connect to the board. While I believe it is useful to have this support out-of-the box for newcomer users or for frequently common use cases one could ask if the goal is to add a conditionally-compiled call to every possible device driver from every possible board, along with the corresponding configuration. It should be noted that it is difficult to avoid that these configurations do not fall out of sync between each other.
While moving most of these calls to a common location (at least at the sub-architecture level of board-logic, ie "stm32") partly improves this situation, board-logic will still have to conditionally include calls (and includes) to this common logic unless something else is added.
The use side
In general, besides the correct maintenance of board support itself, there's a usability aspect which I think it is significant on the user-side, which I think is very important to consider as well. Consider any general-purpose board / prototyping board (ie, any kind of evaluation board which users may have and use to connect off-board devices and use in different ways): if the user wants to test a sensor that is already conditionally-initialized from board logic, this indeed is the easiest scenario which only involves loading up the corresponding configuration and building. However, if a sensor is not yet supported by the board logic the user is forced to modify the nuttx repository. This in itself is not bad for quick experiments but when building up a project it means that: a) the user needs to maintain a fork of the nuttx repository, b) make a custom board which permits using mainline nuttx respository or c) contribute this logic upstream. Going with c), the problem of the combinatorial explosion of board-to-device gets only worse. Going with b) this means that maintenance of the copied board logic becomes a problem for the user. So a) generally seems to only choice which does not become a maintenance problem for NuttX maintainers nor the user. Forking itself may not be a bad choice but one should ask if it is a good workflow to promote.
Anyway, in all three scenarios, the user must modify existing code or add calls to initialization functions and possibly continue the code duplication.
Proposal
The base idea of the proposal is to reduce board logic to the minimum required and maximize reuse between sub-architectures and even architectures if possible. For simpler cases, Kconfig entries should be used to enable/disable optional support of functionality whenever possible (clock selection and configuration is an ideal first step here). For the rest, a combination of hand-written skeleton code plus generated macro definitions (for example, for pin configurations) should be used. To do so a macro definition generator should be written based on an input file which describes the available resources of the board (ie, the various instances of the different on-chip peripherals and all on-board devices) and how they are configured and mapped to drivers and an input file which guides the generation itself (what macros to generate and how).
What I described above is essentially how device are handled using the device tree approach: a text-file (DTS) describes resources in the form of a tree, giving each resource a property. The data structure itself is generic (think XML) and one could give its own semantics depending on use. On Linux the DTS file is compiled into a binary file which is then processed at runtime by the kernel. Zephyr RTOS follows an approach as described above: it generates macros based on a DTS and a YAML (which guides the generation process). I believe this is an appropriate approach for supporting boards in NuttX.
A board could be supported by a set of minimal configuration files, a DTS, a YAML and the skeleton logic which depends on the generated macros. Furthermore, device trees are usually overlaid, which means that a board can only be minimally supported with its on-chip and on-board resources and, later on, a DTS can be overlaid to the previous one, just adding the resource definitions for off-board devices.
Implementation details
To parse a DTS and YAML file is very easy using different scripting languages such as Python. In this case, existing libraries can be used to bootstart this effort. As I understand that Python is not nowadays a tool dependancy on NuttX and may not want to add it, it is also realtively easy to parse a DTS file (I found at least two different libraries with compatibles licenses) and YAML (there are minimalistic libraries for this) or a simpler alternative to YAML.
The tool to be written would parse the DTS and YAML (or whatever other format) and generate macro definitions, include directives and maybe small pieces of very simple code if needed. Skeleton logic could use this generated code in order to conditionally call initialization functions or required glue logic. If various instances of a resource is present (ie, many I2C buses, many sensors on each bus), macros could be used to statically invoke all functions in order to minimize runtime looping.
This skeleton logic should be overridable in some cases where board particularities may require different handling inside glue logic. In other words, this functionality should aid and direct the definition of board-specific logic only when really required for a particular use case.
This proposal would not have to supersede the current system for board logic definition, which would always allow going the "all manual" way. However, for the majority of use cases, this would not be really required. Existing board logic could be slowly migrated to the device tree approach without need to break everything in the process (ie, migrate and test one by one).
First step
Even if this gets positive feedback, the real test is to actually try and do this and see where the shortcomings appear. For this reason, I could start with existing DTS and YAML parsers and focus on which resources would be defined under NuttX, how code would be generated and which skeleton code is needed. This could be tested with a couple boards of different families and could then be tested by other users in different platforms.
Once the mechanism is matured, a native parser/generator could be written in C (if this remains as a requirement) and everything tested for conformity (same exact code is generated).
Final Notes
As I hope I managed to convey, my intention is to make NuttX more powerful, maintainable and user-friendly. I realize the idea could be go a bit against how NuttX has evolved over time but I would greatly appreciate your views on the idea and possible ways to improve it.