Cyanochew editor

[polfeliu]

Hello Everyone,

A few months ago I created a code generation project named cantata (https://github.com/polfeliu/cantata), that generates code for ECUs connected to CAN networks according to the spec a .dbc file. Something similar to cyanobye .yaml files. This is very standard in automotive and while I was doing it I wondered if I could have something similar for SPI and I2C. This is when I discovered cyanobyte. The big difference is that .dbc files typically edited with a GUI, the most known being Vector's CANdb++ editor.

So, allow me to introduce you to Cyanochew! A cyanobyte GUI editor, to create and edit .yaml files according to the spec as fast as possible, chewing through datasheets. Also, byte -> bite -> chew, all puns intended :) I think having a GUI is important to grow the library of supported devices and make cyanobyte something more widely used.

https://github.com/polfeliu/Cyanochew

Right now the project is a minimum viable product and many features are still to be implemented, for example input validation for some types of data. Feel free to open new issues proposing changes or new features, and to exchange ideas. For example, I don't have a solid plan on how functions and extensions should be edited, and for now they are ignored (they won't be removed from files but there is no way to edit them)

One cool thing about Info, I2C and SPI is that the UI is dynamically generated from to the json spec. So it will be very easy to add newer options of cyanobyte. The register editor (Tab Registers, Select register, Edit) was a big challenge but now it's working really well.

[polfeliu]

@Fleker

[Fleker]

This is a really great tool. Thanks for sharing it! Setting up functions are definitely a more convoluted part of the spec in general, and extensions are even more broad and open to interpretation. Having the basic parts to setup the file and create that boilerplate should go a long way.

I can see there's good similarity b/w CAN and I2C in some ways, although CAN may be specced more formally than I2C, it's good that there's interest in the ideas more generally.

[polfeliu]

Thank you! yes, CAN lower layers are definitely more standardized, but the candb standard doesn't cover some weird implementations that some manufacturers occasionally do.

I had some ideas of doing some kind of pseudocode for functions, or an editor based on blocks, simulink like. There are some interesting helpers out there: https://github.com/leon-thomm/Ryven, https://github.com/gangtao/pyflow

One thing that will be a must is the ability to test or plot functions with dummy data.

[Fleker]

Handling edge cases is very tricky, which is an area where defining functions can help massage the data for both CAN and I2C.

A visual editor for functions could be very interesting if there was an easy way to parse that and reformat it.

But ensuring it works through tests is very valuable too. I've thought about having a way to let one define example values from a register to use for simulations. Then you could generate a virtual driver in a console and interact with it through pure logic rather than needing a physical device.

[polfeliu]

I don't know how it would work but being able to define some behaviour of IC's is really interesting. Being able to emulate, also in hw, would be very useful for validation purposes. Its not the case for hobbyist but for professional mcu validation being able to emulate and then introduce malfunctions of the IC is somewhat typical for functional safety applications. Also useful for sensors that capture difficult or expensive to reproduce environments (Light intensity, high voltage, current, moisture, smoke... the list is long)

The functions editor would need to be limited to the functions the specs supports. Then it should be straight forward to parse and deparse.

[Fleker]

I think a simple approach is to define one or multiple example values from an expected sensor (maybe with additional semantic information)

registerA:
    example: 0x43
    # Or several possible values
    example: [0x43, 0x36]
    # Or maybe semantics
    example:
        valid: [0x43, 0x36]
        error: 0x00
        # Custom error keys?
        err_disconnected: 0x77

Then a new template can be developed that substitutes the existing I2C calls with simple returns so that this can be processed in a hardware-free environment (QA, development, CI, etc.)

def read_register_a():
    # return i2c.read(REGISTER_A_ADDR)
    return 0x43 # Obtained from example field

Then in cases like the BMP180, we can still evaluate the functionality of higher-level operations like temperature calculation because all it depends on are register values. If those are mocked, then everything else can be continue to be checked and validated.

def get_temp_celsius():
    reg_a = read_register_a() # Get example value rather than HW op
    # ... perform calcs...
    return temp

[polfeliu]

If I understand correctly, the value of the registers would be calculated when reading or writing from SPI or I2C. In software its no problem but in hardware this has the drawback of requiring a fast calculation to respond, limiting what can be done. I think the emulation should have a state, meaning the emulator should have a region of memory with the registers that it can read and write and the magic should happen behind doors. In Vector CANoe there is something called "environment variables" that describe real values of the state of a simulated car (we should find another name). Then with CAPL/Simulink/... it emulates the behaviour of the ECU.

These env vars could be controlled via a generic interface. (In a SW environment with function calls, in hw with ethernet, usb...)

Then, to actually emulate the behaviour there would be a "worker" (a better name can be found too) that does calculations and sets sig vars and registers/fields doing calculations whenever a event triggers.

emulation:
    envvars:
        temperature: float64
        humidity: float32
    workers:
        workerTemperature:
            triggers:
                - change: "#/emulation/envvars/temperature"
                - every: 200 ms
            function: #Same or similar to regular functions
                variables:
                - somevariable
                logic:
                - somevariable
                    - division
                        -"#/emulation/envvars/temperature"
                        - 128
                 - writeRegister
                      register: "#/register/temp"
                      value: "somevariable"                     

Anyway, I think this could answer both simple and complex emulations in software and in hardware

[polfeliu]

A visual editor for functions could be very interesting if there was an easy way to parse that and reformat it.

There is a slight problem with a visual editor. As far as I understand variables in a same function can be rewritten.

Example: x=0 x=x+1

This does not fit well with the "propagation" logic of block diagrams. Would you be open to limit variables to one assignment? Do you see it as a limitation for cyanobyte?

[Fleker]

In some microcontroller architectures they'll cache register values in a memory map, so repeated queries (especially for read-only) will read from a memory structure rather than call directly. While that can be implemented in a particular template, I have not seen a critical need for the performance gains in my applications.

But the question for an emulator is whether you even want the physical device or want the device to be virtual as well.

I do agree that in a block diagram system the variable reassignment could be a bit of a problem. In standard programming languages it's acceptable except when explicitly marked otherwise. Depending on a particular function implementation, it may be onerous to have to define more variables than necessary which could use more memory than necessary. It may also be hard to validate using a JSON schema.

[polfeliu]

I have little experience emulating SPI devices, but it's something that I think can be useful, also in hardware. To validate functions or the whole driver, it can be easily done in sw. But to test complete applications in another hw platform is something that won't be trustworthy as platforms can have all sorts of implications. I'm not saying leaving sw emulation out of the question, it will be very useful to test drivers while developing. I'm just saying the spec should contemplate both cases.

I have started to dislike the idea of block diagrams for functions. I didn't know how the position and path of the connecting lines would be stored (important to be able to visually understand what a function does) and the reassignment of variables makes it unviable.

I've done some research on text lexers and parsers and it's quite easy. So I'm thinking the function editor will be syntax based. Example:

var a: float32
var b: int16
b = readRegister(registerB)
a = 1 * 2 + 3 + b

With validation and text completer it should speed up writing functions

[Fleker]

I suppose there are two groups for whom emulated devices may be useful:

• Driver spec engineers who may want unit testing on their functions as a double-check
• Product engineers who are using specs in their application who want to test device APIs and have integration tests

For the first group, hard-coding potential register values would be useful for unit test purposes. Having a way to embed test cases could be an extra level of validation.

For the second group, the use-case is potentially less useful due to the hardware quirks as you mention. Perhaps an emulated version with the same API can be useful for software tests, but may not be a primary focus.

Integrating a text lexer and parser may be useful. The YAML pseudocode format is already basically an AST, so you won't need to go all the way to end-code. But I also don't know whether it makes sense to create an alternate programming language. YAML, for its flaws, is universal enough that C, Python, etc. developers can use it. By creating Cyanobyte-lang (name tbd), would that harder to learn?

[polfeliu]

YAML it's universal, but the cyanobyte spec is not and should learnt by the same logic. I think a syntax to develop functions is more readable (and writable) than a YAML file for developing functions. The idea would be that the syntax is exactly as some language, but limited to what is supported by cyano. Python, with type hints I think very easy to understand and most of us are used to it (ignore the "var" in the previous post). Can registers be read on a middle of a operation? Or they have to be read first to a variable?

[Fleker]

A subset of Python could be more approachable, as long as it doesn't delve too deeply into Python syntax.

Can registers be read on a middle of a operation? Or they have to be read first to a variable?

Registers should be stored in a variable before using them later, as different platforms may read them differently rather than returning directly. A read should not appear in the middle of a summation.

[polfeliu]

Perfect,

I wasnt shure of that. Any suggestion for the syntax then? An functiin with readRegister() may be confusing to not be able to place it in an expression

[Fleker]

It depends on how you want to define it. You could introduce some special syntax like:

var a <- register RegisterA

Or something along those lines which is non-standard Python, adding a learning curve but being different enough to prevent it being misused.

Maybe something like

with registerA = a

[polfeliu]

I like this syntax. It opens up the possibility to, someday, also be able to read fields

Selecting by title inside the register

var1 <- field RegisterA.FieldC

or the global name (on #259 we clarified is a unique key)

var1 <- field regAFieldC

I've created a .md with a proposition of syntax for the current function spec: https://github.com/polfeliu/Cyanochew/blob/master/cyanochew/functionSyntax.md Let me know your thoughts :)