search

kernelci / kernelci-project

KernelCI Linux Foundation project documentation
9 stars 26 forks source link

Enhance KernelCI Maestro to Support Numerical Values #426

Open nuclearcat opened 1 month ago

nuclearcat commented 1 month ago

Many of the tests we run yield numerical values as output. For example, fluster tests may produce a "score" value, and boot time tests may return values in seconds or microseconds. Currently, we only support test results as enum values(pass/fail/miss/etc). We can enhance our node model to support additional numerical values, providing more detailed test results.

For reference, see the discussion: GitHub Issue #546.

Objective: The goal of this task is to extend the node model to support numerical values. The task can be divided into the following steps:

  1. Design necessary field(s) in the node model

    • Identify and define the additional fields needed to store numerical values in the node model. @JenySadadia proposed to use "data" subfield.

  2. Add support to LAVA parser (optional)

    • Update the LAVA parser in kernelci-core/kernelci/runtime/lava.py to handle the new numerical fields, if necessary. Probably @pawiecz can implement, or assist as LAVA expert.

  3. Implement data retrieval from LAB results

    • Modify lava_callback.py in the kernelci-pipeline to retrieve and process numerical data from LAB results.

  4. Review and discuss KCIDB schema

    • Evaluate if the KCIDB schema has the necessary fields to accommodate the new data. Propose changes if required.

  5. Assist/submit PR for KCIDB changes (optional)

    • Contribute to the KCIDB repository by assisting with or submitting a pull request to implement the proposed schema changes.

  6. Implement data submission to KCIDB

    • Update the process to submit the numerical data to KCIDB, ensuring the new fields are correctly populated.

KCIDB related changes is more expertise of @JenySadadia i believe.

Steps:

r-c-n commented 1 month ago

Currently, we only support test results as enum values(pass/fail/miss/etc). We can enhance our node model to support additional numerical values, providing more detailed test results.

Identify and define the additional fields needed to store numerical values in the node model. @JenySadadia proposed to use "data" subfield.

These may be two different things. If what we want to support is tests whose result is a number instead of a discrete value (for instance, performance tests), then what needs to be reworked is the result field. If what we want is just to provide an additional field for a number besides the result, then yes, adding a data field is a no-brainer.

nuclearcat commented 1 month ago

Definitely it will be just additional field.