search

RIOT-OS / RobotFW-tests

Includes tests for RIOT based on the Robot Framework
GNU Lesser General Public License v2.1
4 stars 13 forks source link

Add plotting scripts overhead #100

Closed pokgak closed 3 years ago

pokgak commented 3 years ago

This PR adds the script to plot the results of overhead setting and removing timers. Example of how it will look like is attached in the zip file.

overhead_plots.zip

depends on #99

MrKevinWeiss commented 3 years ago

I manually triggered the CI https://hil.riot-os.org/results/robotFW_tests/1/arduino-due/tests_xtimer_benchmarks/report

MrKevinWeiss commented 3 years ago

Plots look good

MrKevinWeiss commented 3 years ago

Maybe you can rebase again and I will rerun everything. We should get this in before the robot v4 update.

pokgak commented 3 years ago

Rebased to master

MrKevinWeiss commented 3 years ago

Hmm something odd is happening with the frdm-k64f board... https://ci.riot-os.org/hil/blue/organizations/jenkins/robotFW_tests/detail/robotFW_tests/29/pipeline

I will rerun as it looks like maybe just a temporary interruption.

MrKevinWeiss commented 3 years ago

Hmmm I don't see the plots...

https://hil.riot-os.org/results/robotFW_tests/30/overview

Only the initial 2.

pokgak commented 3 years ago

I'll take a look at this today.

pokgak commented 3 years ago

I suspect this is due to how the page is built using XSLT. This line seems like it hardcodes the name of the files that are included on the page.

@ozfox instead of hardcoding the names in a config file, is it possible to just go through all files in the includes folder and include them on the page?

ozfox commented 3 years ago

@ozfox instead of hardcoding the names in a config file, is it possible to just go through all files in the includes folder and include them on the page?

That is not possible with XSLT, all input data has to be read from xml files or passed to xsltproc as a parameter. That's the reason why it's set it in the config file as it separates it from the code, at least.

The alternative would be to pass a --paramor --stringparam when calling xsltproc in RobotFW-frontend/scripts/xsltprocw.sh. This script could check the includes/ directory and pass the result as a param.

I don't know if this is currently worth the effort, as I currently don't expect the plots to be changing very often. It's more a problem of RobotFW-frontend missing a meaningful README, which is still on my list.

What's your opinion on this?

@MrKevinWeiss I'll fix the pages.xml config later today, the plots will appear afterwards

ozfox commented 3 years ago

Changed plot names with https://github.com/RIOT-OS/RobotFW-frontend/commit/c98495405e4b9ed6b0ce9f110175996708a6950a

Let me know if you would prefer the --param solution and I'll create a Issue for it.

MrKevinWeiss commented 3 years ago

We can leave it hard-coded for now

MrKevinWeiss commented 3 years ago

https://hil.riot-os.org/results/robotFW_tests/31/overview

Things are looking OK now. I will verify all the plots.

MrKevinWeiss commented 3 years ago

They look good. Failed tests don't generate the graphs but that is a later problem... Any last words @pokgak or @ozfox ?

pokgak commented 3 years ago

Nothing from me.

ozfox commented 3 years ago

Looks good to me. I added Display error or notification for missing plots on report pages #5 to the RobotFW-frontend repo.