Concerns around portability and openness of solutions / projects

[madchutney]

An expected characteristic of a solution / project format and tooling is that it should be portable and open. One user should be able to share their project with another user and they should be able to use it on their system without modification.

A common way of doing this is by cloning a repository, so they have exactly the same source. It would also be reasonable to expect that the users have the same tools installed. However, it would not be expected that the two user's systems be identical.

The following aspects of the proposed system could make sharing solutions / projects more difficult and require more that just the software repository to be shared.

1. The currently installed packs impact what software is built and potentially what can be built.
2. A system level defaults file (installed alongside the Open-CMSIS tools binary) can impact the solution and project definition.

Installed Packs

If the packs are not completely specified in the solution file the project manager will generate build instructions (cprj) files based what is currently installed on the system. This means that different users may end up building different software based on what is installed on their system. Also potentially meaning that a user will have to go through a dependency resolution process, not knowing which packs should be used.

For the online use case, this is also a problem as all packs will be available. Therefore, if the packs are not specified, then the project manager will most likely consume a lot of memory loading all the packs and may still not be able to be able to resolve which pack to use.

Recommendation: Specify all packs required in the project / solution, with an optional version number or range.

System Defaults

A system defaults file means that files outside of the solution / project repository will impact how that solution / project will be built. It is very easy for someone to forget that they have modified a system file, which is likely to lead to both projects being shared that don’t work in all environments or users potentially setting up their environment so that certain projects are unbuildable.

This also substantially impacts the openness of the format as it is unreasonable to expect other tools or systems that work with Open-CMSIS to find the installation location of the Open-CMSIS tools. Which is what will be require in order for them to be able to accurately process the solution / project files.

Recommendation: Remove system level defaults from the standard

[ilg-ul]

FYI, for reproducibility reasons, the xPack Build Framework requires all projects to have their dependencies fully specified, with versions.

Even more, in addition to project wide dependencies, it is possible to define dependencies specific to build configurations.

This allows to conveniently build multi-platform applications, and it is particularly useful to build and run multi-platform unit tests.

Dependencies refer to:

• source packages, which, by design, include only a single component, with a single version; there are no huge packages, like the existing CMSIS packs, with lots of libraries, each with their own version;
• binary packages, like toolchains, cmake, meson, ninja, etc

The workflow is highly structured, and before starting the build an explicit xpm install step must be performed, to satisfy all dependencies.

After this step, the build expects to find all dependencies locally, and does not need to rely on anything else installed in the system folders. This is particularly useful for running tests on CI servers.

You can take a look at such a real world multi-platform project using the xPack build framework:

• https://github.com/micro-os-plus/micro-test-plus-xpack/blob/xpack-develop/package.json

An example of a GitHub Actions run of the unit tests on all platforms can be seen at:

• https://github.com/micro-os-plus/micro-test-plus-xpack/actions/runs/1983066733

If you think that such design principles can also be applied to Open CMSIS, don't hesitate to ask for details.

[fred-r]

We have a trade-off to find between:

• portability : easy to reuse a project setup with different packs (because different HW or because bug fix, or because 2 packs can provide 2 sets of components fulfilling the requirements)
• reproducibility : making sure to have the same build outcome

I think YML is good for portability because it gives flexibility : specify versions, versions ranges, minimal versions, no version at all... [Cvendor::] Cclass [&Cbundle] :Cgroup [:Csub] [&Cvariant] [@[~ | >=]Cversion]

For reproducibility, we clearly need a "lock file" summarizing:

• the packs and components
• the compilers toolchain: CMake, Ninja, AC6/GCC
• the settings

Then, I would say that the lock file could be used to trigger warnings and information but should not prevent a build to be made with different elements as long as the conditions are satisfied.

[madchutney]

Thanks for the responses, they are both very helpful.

@fred-r I like your suggestion. The separation of concerns looks good and seems also to align with the xPack approach suggested by @ilg-ul:

• "intent" e.g.I want to use a compatible pack in this version range
• "reproducibility" e.g. build this exactly as it was built before.

Having a separate lock file that can be regenerated based on the current solution and installed pack information would go a long way towards that.

@fred-r I have a question on your comment:

portability : easy to reuse a project setup with different packs (because different HW or because bug fix, or because 2 packs can provide 2 sets of components fulfilling the requirements)

I'm trying to understand the use case where the same project would be used for different HW or with different components. The solution file allows different packs to be specified for different target-types and build-types (using the for-type and not-for-type). Would this feature allow packs to be specified and still support the needed flexibility, without the ambiguity in pack choice?

[fred-r]

@madchutney : basically, my intention is to avoid specifying packs in the csolution.

I would like to keep my csolution and cproject HW-agnostic.

So I would specify:

• use this IoT application ==> should be HW-agnostic by nature
• use this connectivity middleware ==> should be HW-agnostic by nature
• use this driver ==> here it depends on the device we select

But, I want that based on the device selection the conditions allow the tool to select the appropriate driver from the appropriate pack.

This is of course applicable mainly for projects provided "as example". And the lock file allows to determine how to achieve perfect reproducibility if needed, but at least the csolution/crpoject does not prevent me from switching easily to another HW or driver version or another driver provider.

For projects that need to go in production, then probably I will specify the packs and exact components version in csolution/cproject. But, this is not sufficient, I still need the lock file to have the exact toolchain (CMake/Ninja/Compiler) and maybe other information (like the exact path to the packs, maybe a checksum of the source code or whatever is needed to prove that my software will be regenerated as expected).

Also, another point is what ARM is promoting : a multi-HW csolution. At ST, we are not fond of it, but the spec allows it. To implement this, you can specify everything per target-type or you can have a "loose" specification of the packs and components (conditions resolution based on selected device do the job of taking the appropriate components).

Regarding the reproducibility, another dimension we need to take into account also is the standalone project. If I work with a standalone project, where all used packs have been replicated locally "under root of the project", then when I regenerate the project the tool should consider the same set of packs (the ones copied locally). SO, even if I do not specify the packs explicitely, I see no reason why the tool would not select the same artefacts (as the list of possibilities is "frozen/archived" in the local project).

[ilg-ul]

I'm trying to understand the use case where the same project would be used for different HW or with different components

Multi-platform tests.

I don't know if this is of interest for the CMSIS pack design, but xPacks are by design multi-platform, and being able to run the same tests on multiple platforms with different build tools (like toolchains) was a hard requirement in the xPack design.

For example the tests for my portable projects already run as native processes on macOS/Linux/Windows, and via semihosting on STM32F4DISCOVERY and mps2-an386 (using QEMU). I also plan to run them on Cortex-M0, but QEMU does not provide it yet. And after some details will be sorted out in QEMU, it is also planned to run the tests on 32/64-bit RISC-V devices.

As you can imagine, in order to build the tests for different platforms require totally different components and different build tools (toolchains, cmake, meson, ninja, etc).

These differences are specified via pairs of debug/release build configuration for each platform, each with specific dependencies.

You can take a look how a STM32F4DISCOVERY test is configured:

• https://github.com/micro-os-plus/micro-test-plus-xpack/blob/255d4e5a21e7dfa04e06c3e1e45992bb07cef710/package.json#L657-L740

In the same file you can see the dependencies for the mps2-an386 platform, and for the native tests, each with different packages.

A GitHub Action run of all tests on all platforms can be seen at:

• https://github.com/micro-os-plus/micro-test-plus-xpack/actions/runs/2026948280

I'd be interested to know if there are other solutions more convenient than the xPack Build Framework, since I could not find any.

I think that being able to integrate CMSIS packs in xPack projects (which are neutral to platforms and build systems) would be an important gain.

Also I think that being able to publish the CMSIS Packs on existing public servers like npmjs.com or GitHub npm would be a great feature.

FYI, the xPack Build Framework is able to consume packages from any location, like local folders, local/remote tgz archives, Git repositories and npm packages repositories.

This is a major advantage, since anyone can publish packages without requiring the blessing of a central authority, and also reduces the risks for out of sync indexes, as I continuously encounter now with the central CMSIS Packs index which does not reflect the actual package contents.

There are many details to consider if you want to achieve portability and openness, I already spent quite a lot of time to address them.

[fred-r]

@ilg-ul : in your example you specify by HW all the dependencies with exact versioning. I understand the interest for a product but for examples I would prefer a "loose" definition allowing me to switch easily to new packs without updating the project definition.

So, whatever the target I only say: I need an UART driver. Then, based on conditions, the proper component gets selected.

[ilg-ul]

for examples I would prefer a "loose" definition allowing me to switch easily to new packs without updating the project definition.

Sure, npm allows to specify any such loose semver definition, just that for reproducible builds (like unit tests) exact versioning is a requirement.

• https://docs.npmjs.com/about-semantic-versioning#using-semantic-versioning-to-specify-update-types-your-package-can-accept

However I would avoid expressions that match any major release, since this increases the chances of examples to fail the build.

[fred-r]

In fact, I want the possibility to be even more flexible than this :-) Maybe this won't fly but basically, I want to say : this application can work with any UART driver from ST. Then, if the end-user tries the application with STM32U5 maybe it will be driver v1.1, if it is STM32F4 maybe it will be driver v3.12.

What I want to avoid is a situation where the end-user wants to try something (at his own risks maybe) but my conditions prevent him from trying.

Of course, users who cannot accept this trade-off can specify exact versions (or ranges).

But anyway, for reproducibility (like tests in ACI), I really think versions are not enough, the tooling matters too.

So I would like to keep the openness with the current possibility (not specifying packs and versions).

And when reproducibility becomes key then:

• freeze the packs and versions
• get a lock file to also have other dimensions (like tooling or compiler switches in configuration files...)
• maybe work in a standalone project to make sure my packs are the "ones local to my project" so not modified in any manner

[ilg-ul]

But anyway, for reproducibility (like tests in ACI), I really think versions are not enough, the tooling matters too.

Definitely. That's the reason why in xPack projects, in addition to dependencies to source library xPacks, there are dependencies to npm applications (portable by definition) and binary xPacks (cross-platform by design).

For example:

• https://github.com/micro-os-plus/micro-test-plus-xpack/blob/255d4e5a21e7dfa04e06c3e1e45992bb07cef710/package.json#L37-L41
• https://github.com/micro-os-plus/micro-test-plus-xpack/blob/255d4e5a21e7dfa04e06c3e1e45992bb07cef710/package.json#L770-L781

Thus, in order to build an xPack project and its tests, the only prerequisite is node, which currently is almost ubiquitous on all modern platforms, together with npm. I don't know how Open CMSIS will end up, but for CMSIS Packs, this aspect was not even considered.

But, returning to your question, I think that we are dealing with different things. What you basically ask for is a metadata specification for a template engine, able to create projects. I think that this is a completely different thing and should not be mixed/messed with the specifications for the metadata defining a project (dependencies, build configurations, actions, properties, etc).

In the current xPack design, things are really separated, the metadata project is stored in package.json (with a functional example available in the previous links).

Templates are totally separate projects, which have both a CLI interactive interface and a programatic interface, that can be invoked from a terminal or a GUI environment, and result in functional projects. Actually in this regard the xPack design is very loose, there are almost no limitations, and any solution acceptable for the template author is fine.

[fred-r]

Metadata for a template engine ==> not really, let's say applicative component with generic conditions allowing to create easily projects

Projects ==> even for a project, I want that it is easy to try it with: a- another set of source files (different packs or versions) b- another hardware

a => updating my CMSIS_PACK-ROOT should be enough, I do not want to rework the packs and versions in the project (just like when you have a project in git, if you update the git repos then your project takes the new code into account without reworking the project definition)

b => here I need to modify the project file to indicate the new HW (or to create a new project in ST view). Still, if I can have very few modifications to do it is great : no versions update, just a new target-type

This is the flexibility I'd like to give to the users.

[ilg-ul]

Metadata for a template engine ==> not really, let's say applicative component with generic conditions allowing to create easily projects

Name it as you wish, I mean a set of definitions and a set of rules implemented by an engine, that, based on some user input (scriptable and/or in a GUI), either create a new project from scratch, or update an existing project.

Metadata for project

• definitions that allow a tool to satisfy the dependencies; this means downloading all source libraries and tools required for the build
• definitions of multiple build configurations, each with their dependencies and portable scripts to perform different build steps

Some solutions (like CMake) tend to mess things and integrate the dependency management into the build, but I think this is not beneficial, and dependencies should be satisfied in a separate step before starting the build.

---

In my opinion creating/updating projects from templates, satisfying dependencies and building projects are separate things (any resemblance with the xPack design is not purely coincidental!).

If you can achieve this with the current Open CMSIS proposals, it is ok.

[ReinhardKeil]

Thanks for all the feedback here, but basically I believe we need two different modes of working with csolution.yml and cproject.yml files.

1. Create where the user has full access to all available packs.

2. Freeze where the user fixes all the packs that are used in a project. This allows to share/store the project.

This would be similar to what uVision offers today: https://www.keil.com/support/man/docs/uv4/uv4_ca_packselect.htm It could be therefore supported by an IDE that creates the csolution.yml/cproject.yml files.

[ReinhardKeil]

Is there anything that we should discuss here again?

[madchutney]

I don't think we have discussed the use of cdefaults.

[VGRSTM]

@madchutney no we don't ... at a point such file was close to disappear Thanks some reproducibility concerns but sounds back now ... A bit unclear status today to me too.

[ReinhardKeil]

A potential usage of cdefault.yml is to specify a default toolchain. For example:

• GCC (default on VS Code)
• ArmClang (default on Keil Studio)

• IAR (default on IAR tools)

  This would enable seamless retargeting of csolution.yml based projects between different toolchains.

Yes, agreed that it has some testing challenges, but I believe it's worth exploration. To achieve it, such a cdefault.yml file could be part of the "./etc" directory.

[ReinhardKeil]

Discussion should continue here https://github.com/Open-CMSIS-Pack/devtools/issues/315 as this issue is a bit diluted.