dist/tools: Update to robotFW v4

[MrKevinWeiss]

• Update requirements
• Adapt output_to_xunit for new versioning
• Improve example test
• All make command to run example test

[MrKevinWeiss]

ping @ozfox

[ozfox]

Looks promising, I'll check what https://github.com/RIOT-OS/RobotFW-frontend says.

[MrKevinWeiss]

I guess we have some semantic issues with what is an error.

Currently we count an error as the deprecation notice of non-critical tag that should be skip as well as when we throw a warning (warn on fail or something).

To me these are warnings. With this script we now just pass tests that have the errors (or warnings). It would be possible to adapt the script to catch the warnings and write them as xunit errors with the type as warning and the message.

This means that the frontend script could then show passed, skipped, failed, and warnings on the overview.

This would be useful with something like the spi timing checks, we do not want to skip them, nor do we want them to fail the test. If they just pass then we don't know that they are out of spec (which is OK). Thus, if we can having a warning saying they are out of spec, check the values if this makes sense, maybe that would be best.

This would be what I would imagine as non-critical...

[MrKevinWeiss]

After talking we decided to drop the warning idea.

We will only have skipped, pass, fail, or error. The error results will only occur due to some underhanded exception (keyboard interrupt or device unplugged or something).

When using the Run Keyword And Warn On Failure an error will be counted in the xunit errors header but not error flags occur in the actual test. We can sort that out if we end up doing something actionable with the errors...

[MrKevinWeiss]

Here are the results: https://ci.riot-os.org/staging-hil/blue/organizations/jenkins/robotFW_tests/detail/robotFW_tests/10/pipeline

https://hil.riot-os.org/results/robotFW_tests/10/overview

Note that this is generated with https://github.com/RIOT-OS/RobotFW-frontend/pull/6