dist/tools: Add better parser for finding env

[MrKevinWeiss]

This adds a env_parser python script that allows better control over how to show the env. It allows a human readable version to be printed to the console and an xml to be generated.

This should allow metadata to be added to the ci test results. The print_environment.sh is updated to use it but should be transparent the current ci system.

Check the usage of the env_parser with

./env_parser.py --help

To collect all results in an xml theresults_to_xml.sh has been updated and added to this repo.

To use from RobotFW-tests base.

./dist/tools/ci/results_to_xml.sh build/robot

This will generate a collection of the metadata and tests stored in build/robot/robot.xml

[ozfox]

Generally this looks very promising! Goodbye RobotFW-scripts, you won't be missed.. :)

I only have two comments on this:

1. Is it necessary to have two files as metadata.xml contains the same content as robot.xml? I don't care if it's one or two files at the end but I would not duplicate the data.

2. Differently than expected there is a lot of metadata now (which I like). I'd prefer a hierarchy then like the following which would also prevent the unpleasant mixed uppercase/lowercase element names.

<?xml version='1.0' encoding='UTF-8'?>
<metadata>
    <riot>
        <commit_id>6f271b7d3ebd20b298d7c20dfb7a250012985ea1</commit_id>
        <branch_name>master</branch_name>
        <commit_timestamp>2020/09/01-15:15:55</commit_timestamp>
        <version>2020.10-devel-1164-g6f271b7d3</version>
    </riot>
    <robot>
        <commit_id>32cb7e9ef3d03afa7170d216c63d11b0c8041139</commit_id>
        ...
        <commit_timestamp>2020/08/28-13:48:26</commit_timestamp>
    <robot>
    <environment>
        <PATH>/usr/local/bin</PATH>
        <GJS_DEBUG_TOPICS>JS ERROR;JS LOG</GJS_DEBUG_TOPICS>
        <GJS_DEBUG_OUTPUT>stderr</GJS_DEBUG_OUTPUT>
    </environment>
    <python>
        <modules>
            <pal>missing</pal>
            ...
            <deepdiff>missing</deepdiff>
        </modules>
    </python>
    <os>
        <operating_system>"Arch Linux" </operating_system>
        <kernel>Linux 5.7.12-arch1-1 x86_64 unknown</kernel>
    </os>
    <toolchains>
        <native_gcc>gcc (GCC) 10.1.0</native_gcc>
        ...
        <clang>clang version 10.0.1 </clang>
    </toolchains>
    <devtools>
        <arm-none-eabi-newlib>missing</arm-none-eabi-newlib>
       ...
        <coccinelle>missing</coccinelle>
    </devtools>
</metadata>

[MrKevinWeiss]

Is it necessary to have two files as metadata.xml contains the same content as robot.xml? I don't care if it's one or two files at the end but I would not duplicate the data.

Yes because the robot collection script just combines different xml files. I wanted to a keep the similar approach, it gathers all the xunit files and gathers the metadata.

This also allows more of a standalone operation. If you just want to save the metadata you dont need to call the full script.

With that note maybe the script can just capture a printout if it is space that you are worried about.

In the CI we can also just not archive it.

-> Differently than expected there is a lot of metadata now (which I like). I'd prefer a hierarchy then like the following which would also prevent the unpleasant mixed uppercase/lowercase element names.

This just means we should set the flatten flag to false :)

[ozfox]

ACK, this is what I need!

[ozfox]

As asked, here is the preferred date format: 2002-05-30T09:00:00 Here is the doc https://www.w3.org/TR/xmlschema-2/#isoformats

[MrKevinWeiss]

It seems the python modules are needed in root of the CI node (gitpython and xmltodict).

This isn't the only problem... The repos are not actually cloned onto the nodes, just the files are copied so using git commands on the nodes don't work.

I think I will do the following:

• make each group require a flag instead of a shutoff flag
• Do not require git module
• Add exceptions
• Maybe use logging to through exception messages

[MrKevinWeiss]

:partying_face: