musamaanjum
commented
3 weeks ago CI systems like 0-day and Syzbot are successful as they provide the entire information to reproduce the failures. Anybody should be able to use the email/dashboard data to investigate and fix the issue. We should try to follow the same motto that the entire information must be present in bug reports/dashboards. We can add information in a step by step process.
One thing that I've noticed recently in legacy KernelCI is that the information regarding used rootfs isn't easily available. So its very difficult to fix issue as you don't have exact reproducer environment.
This issue is to initiate an open discussion regarding the possibility of providing users with more comprehensive information about the specific configurations of test suites through the KernelCI nodes.
Tests may be executed with different parameters across different platforms (depending on factors such as hardware configuration). Thus, it's important to provide details on the configuration/parameters used on a specific test run. This enables users to know how the test was conducted on each platform and to easily identify if a test failure correlates with the specific configuration in use.
This may become particularly relevant with the addition of support for running kselftests with parameters (see https://github.com/Linaro/test-definitions/pull/511).
Some possible solutions:
Some test suites can be harder to describe than others, so I feel like the challenge is to find a way to effectively describe the test configuration that works for different test suites (maybe a link to the test job template used is enough?).
Any thoughts on this?