How to support multiple configurations of Nanostack in mbedOS ?

[hasnainvirk]

Currently There are 3 binaries present in mbedOS. One for each compiler. And each binary have a generic configuration, i.e., mbed-os.cfg (which includes everything, hence increased binary size)

However Nanostack can be configured with many different configurations which potentially decreases binary size for certain stripped down variations of binaries.

In mbed-OS3 CMakelist was used to pick certain binary with a given configuration pertaining to a specific compiler. There is no such mechanism in mbed-OS5.

How and when such config mechanism will be implemented ?

@c1728p9 @sg- @kjbracey-arm

[maansaari]

@simonfordarm @sg-

[sg-]

I don't understand why the stack is compiled to pull in both. Surely there is an entry point into the stack for initializing 6LoWPAN that is different from initializing Thread?

[hasnainvirk]

@sg- Nanostack can be compiled with various configurations. The current available binary was compiled with configuration mbedos which contains all possible configurations (router, border router, host for both 6LoWPAN and Thread) so that the application can have freedom of choice.

[sg-]

If these are compile time configurations it sounds like the stack doesn't lend itself well to being released in binary form. How many configurations are there and why compile time options not just dead code eliminated by the linker when not used?

[kjbracey]

We've been making some effort to push in that direction - we don't particularly want to have multiple configurations - but it's not that straightforward.

Partly because it's a network stack. A large amount of code and complexity in code is triggered by network activity. A lot of the "entry points" come from packet reception - there aren't that many code entry points by comparison. And if "feature X" is enabled, then we need a more complicated version of some core routine Y, or a bigger version of core structure Z.

And no matter how much effort we put into shrinking the generic build, it can never be as small all as a cut-down build. That's inevitable. A single-configuration build can reduce run-time tests to compile-time tests.

(Compare "target_is_XXX()" macros in the Linux kernel - that macro is a run-time test if a multi-SoC build, and a compile-time test that can eliminate code if built for a single SoC. The run-time test can never be as efficient. We have a similar tests like "is this interface in Thread mode" in chunks of the core, to do things in different ways).

Anyway, multiple configurations is not a particular distribution problem - the build process happily automatically spits out all 6 (or was it 8?) configurations (times 2 for toolchains). We just need a mechanism to select one, as we do in mbed OS 3, and in other Makefile/project-based systems.

Had some discussion with Hasnain on this - my suggestion was to extend the existing "only include a directory with a special prefix conditionally" mechanism.

At the minute, the tools have special directory handling for "FEATURE_XXX", "TARGET_XXX", "TOOLCHAIN_XXX" (any others?).

My suggested extension is "MBED_CONF_XXX"/"MBED_CONF_XXX_YYY".

MBED_CONF_XXX directory is included if configuration option XXX is set.

MBED_CONF_XXX_YYY directory is included if configuration option XXX is set to YYY.

In the case of underscore ambiguity, each match possibility is tried - so MBED_CONF_XXX_YYY_ZZZ would be included if either "xxx" is set to "yyy.zzz", "xxx.yyy" is set to "zzz" or "xxx.yyy.zzz" is set.

Those forms seem consistent with existing behaviour - the basic name matches the C define for the config option, and you are making a multi-way selection based on a final "_XXX".

The ambiguity resolution permits an option to use either "select one" or "select multiple" forms of config by changing the JSON without needing to change the directory names (switching between"xxx.mode"="a" and "xxx.mode.a"=true, "xxx.mode.b"=true).

If that multiple match rule is "too clever" then maybe use a different final separator. I believe "MBED_CONF_XXX=YYY" would be a legal directory name in all platforms. But seems a bit inconsistent with the other special directories.

@bogdanm, for the config relevance.

[bogdanm]

MBED_CONF_XXX_YYY directory is included if configuration option XXX is set to YYY.

That can be done with some effort, yes, even though it'd make the tree structure even more confusing than it already is. I'm a huge fan of not using the config system to select build components, but if the current TARGET_xxxx mechanism is not enough, I don't think there's a better way to do it, sadly.

[kjbracey]

This approach is kind of specific to needing configuration from binary builds. It's the best approach I can think of in this system. Using "TARGET" would seem like abuse. Maybe it could be done as a FEATURE? FEATURE_NANOSTACK_XXX? Feels wrong though, as surely features should be additive, not a single choice. I guess if it's documented as "enable only one"?

We have to reduce configuration choice to "select overall config set named X", as we can't produce enough binaries for any arbitrary permutation of config options. But that technique is not generally ideal, as it does require that the app knows the overall config to use.

For components with source, it's usually better to provide a proper set of independent "lib.feature_x"=true options - eg that's what mbed TLS should be doing, rather than just having a JSON that lets apps point to a config file. The fact that multiple components each need a different set of functions, and we need to build an mbed TLS library with the union of the requirements of all modules, has been repeatedly painful - apps can't be expected to know that set.

For Nanostack, configuration hasn't been problematic so far, as unlike mbed TLS it's not a "utility" library with multiple independent users - the choice is a rather more fundamental "what's the function of this device?" question, which is more suited to central knowledge from the app.

[sg-]

We just need a mechanism to select one

What are the build options suggested to be exposed? 6LoWPAN (non-routing) and Thread (routing)?

[maansaari]

If we think build options from our customer point of view at least followings are in frequent use: 6LoWPAN node (routing) + Border Router Thread router node + minimal node(no routing)+ Border Router

[sg-]

So that is 5 configurations (features) for 3 compilers so 15 binary releases??

[jupe]

there is a lot of more configuration available but those are probably most used configurations.. @maansaari: should there be also non routing 6lwp node as well? what about different security related configurations,,,?

[kjbracey]

So that is 5 configurations (features) for 3 compilers so 15 binary releases??

No, 1 binary release, containing 16 binaries, for mbed OS 3 (2 toolchains x 8 configs). It doesn't make sense to say "15 releases". We're just hitting one build button. And hence giving binary users access to 8 of the available configs they could get via the source release.

https://github.com/ARMmbed/sal-stack-nanostack/tree/master/source/TOOLCHAIN_YOTTA/armcc

• Generic
• 6LowPAN-ND border router
• 6LoWPAN-ND router
• 6LoWPAN-ND host
• Thread border router
• Thread end device
• Thread router
• Thread test harness

The absolute number isn't really a big deal - it's all automated. We had this all in place and working smoothly for mbed OS 3.

Yes, building 16-24 binaries up-front is a bit silly, but on the other hand we're building them so the end users don't have to :) Think of all that aggregate compiler time saved!

Personally I don't think we've got an excessive number of configurations - compared to the thousands of individual config options most software has, and the billions of possible combinations available for something like mbed OS or the Linux kernel, having 8 combinations is pretty slim.

The inefficiency arises from the fact we're forced to release as binary.

[sg-]

Can you help me better understand the need for all these configurations? If the aim is to decrease the binary size, what is the expected target size for each configuration. I get the feeling there will be lots of overlap where a handful of different configurations don't have a meaningful size difference warranting a configuration for release in binary form. TLDR: I think that list can be consolidated if for memory reduction but would like to see the real numbers so we make the right decision.

[hasnainvirk]

@sg- I think you should review Kevin's previous comments. The basic point here is that "We need a mechanism to choose a particular configuration from our binaries". For refrence, please check the comments above or check how it was build in mbedOS-3. I think it would really help if you can setup a short skype call with Kevin and me.

[kjbracey]

I'd just say that the final number of configurations is not going to be 1, even if we did consolidate. So we need the mechanism.

Looking at that list, Thread test harness seems to be redundant, and conceivably there's no huge difference between Thread router and border router. But all the other configs are pretty distinct, so I don't see it being less than 6.

The two different "end device" forms in particular are susceptible to huge amounts of optimisation - removing large chunks of the generic IPv6 core, getting rid of routing and address resolution functionality, and replacing them with "just send to IEEE 802.15.4 coordinator" logic.

[sg-]

FEATURE_NANOSTACK (replaces FEATURE_IPV6)
    ○ NanostackInterface and headers
    ○ FEATURE_NS_6LP_HOST
        § TARGET_CORTEX_Mx
            □ TOOLCHAIN_GCC
            □ TOOLCHAIN_ARM
            □ TOOLCHAIN_IAR
    ○ FEATURE_NS_6LP_ROUTER
    ○ FEATURE_NS_6LP_BORDER_ROUTER
    ○ FEATURE_NS_THREAD_HOST
    ○ FEATURE_NS_THREAD_ROUTER
    ○ FEATURE_NS_THREAD_BORDER_ROUTER
    ○ FEATURE_NS_FULL