google / cyanobyte

Machine-readable datasheets
https://cyanobyte.dev
Apache License 2.0
79 stars 31 forks source link

Add examples to schema for registers and functions (Schema validation) #271

Closed Fleker closed 3 years ago

Fleker commented 3 years ago

Branching off of #267, aid in the development of high-quality schemas by allowing schemas to specify example values for registers and functions.

This will provide:

  1. Improved documentation
  2. Ability to perform verification of logical functions w/o hardware (unit tests)
  3. Allow for logic-only emulation of devices for platform integrations (low priority)

This issue itself will not focus on emulations of devices.

Spec changes

This change will add a new set of fields to the register objects:

example:
    valid:
        - 0x43
        - 0x36
    errors:
         BufferOverflow: 0x10
         OtherError: 0x11

The will be two child fields. valid will contain a list of example fields that may be returned during nominal operation. error will be a map of custom error names (key) to their returned result (value).

The second change will come to the function objects:

functions:
    temperature:
        computed:
            asCelsius:
                example:
                    valid:
                         - 16
                         - 20
                    errors:
                         OutOfBounds: 55
                 variables:
                     # ...

Similarly, each example field will be added to a particular computation. It will have some valid entries as well as a map of error types with returned results.

Unit tests

A new python-unittest.py template can be used to generate device-specific unit tests. Modeled off of the existing Python templates, the register read functions will be replaced by hardcoded entries.

def read_register_a():
    return 0x43 # example.valid[0]

These will be processed by the function execution, defined as tests. Assertions will run at the end to see if the function matches one of the valid examples.

def test_ascelsius(self):
    # Run some logic
    variable = read_register_a() # Get the hardcoded 0x43
    # Additional logic
    returnVal = ... # Whatever is normally computed
    expectedVal = 16 # asCelsius.example.valid[0]
    self.assertEqual(expectedVal, returnVal)

Finally, running this unit test python3 -m unittest /path/to/test.py will verify if the results match.

Templates

These fields can be added to doc.md, datasheet.tex, and webpage.html to provide additional register-level documentation.

They can also improve code documentation, like raspberrypi.py:

def set_{{key.lower()}}(self{% if register.length > 0 %}, data{% endif %}):
        """
{{utils.pad_string("        ", register.description)}}
+        Example value is {{ register.examples.valid[0] }}.
        """

Validator

This can be integrated into the existing Cyanobyte validator tool if the schema includes examples (if not it will generate a file with no tests).

The validator will receive new flags to more precisely tune its output and execution.

$ cyanobyte-validator --validate-schema /path/to/files.py
$ cyanobyte-validator --unit-test /path/to/files.py

By default the tool will do both, as well as any future validation operations. Internally, the existence of any of these flags will cause all flags to be checked. Technically you could use both flags --validate-schema --unit-test /path/to/files.py ahead of a potential third validation technique.

Fleker commented 3 years ago

image