Add `KaNN` format support

[zylfried]

Hi 👋,

We've added support for .kann files to visualize KaNN graphs, a format developed by Kalray Inc. for accelerating ML computations with our MPPA processors.

Compatibility with Netron is essential for both our employees and customers but adapting an existing format do not address the specific requirements for handling KaNN's images and layers, among other things.

kann.js, kann-metadata.json, kann-schema.js have been added in source/ as well as kann.fbs and the script to process it in tools/. Examples are also available in test/models.json.

Could you please review our pull request?

Let us know if you have any comments or need further information.

Thanks!

[zylfried]

but adapting an existing format do not address the specific requirements for handling KaNN's images and layers, among other things.

Can you elaborate on what these requirements are? For example, would it be possible to expose these as operator extensions or metadata in ONNX instead of introducing yet another graph format? What process is used to add this information to KaNN files?

First, the current ONNX implementation doesn't meet the modularity needed to fully address KaNN's data type requirements. Additionally, not all KaNN layers have corresponding ONNX equivalents. For instance, custom layers like cluster_split or specific attributes of KaNN’s convolution layers don't exist in ONNX. But most importantly, KaNN treats tensors as equivalent to layers, which necessitates a shift in how they are categorized. In KaNN, layers and tensors need to exist at the same hierarchical level, with Netron’s typical tensors replaced by arcs representing subviews of KaNN’s tensors.

It's important to clarify that KaNN is an inference engine similar to TensorRT, so the graphs we handle are more low-level than ONNX models. We have our own internal model format, and we'd like to use Netron for visualizing it but the difference in structure and purpose makes adapting existing formats insufficient to meet KaNN’s specific needs.

[lutzroeder]

the modularity needed to fully address KaNN's data type requirements

Can you provide an example or such a type or modularity need?

not all KaNN layers have corresponding ONNX equivalents

ONNX can act as a container with an entirely custom set of operators provided via a "domain".

KaNN treats tensors as equivalent to layers

Can you elaborate in this. What is an "Arc" and is there some context why this is needed?

[zylfried]

Indeed, we could extend ONNX to support KaNN’s data types and layer types, but the core difference lies elsewhere. KaNN’s purpose is not to introduce yet another framework for neural network design but rather to translate models from popular frameworks into a lower-level, processor-executable format. The .kann graph we aim to visualize in Netron represents an intermediate stage of model optimization and refactorization. This stage is then compiled into processor-level kernels, which is the final usable format but could not be visualized in Netron, the intermediate format being therefore used for visualization purposes only.

KaNN treats tensors as equivalent to layers

Can you elaborate in this. What is an "Arc" and is there some context why this is needed?

Unlike ONNX, where layers and tensors are distinct (layers as nodes and tensors as arcs), KaNN manipulates tensors as individual graph nodes, much like layers. The input/output relationships between these nodes are then not whole tensors but subviews (smaller parts of the tensors). These subviews correspond to the arcs in our graph, with an Arc being a connection between nodes, as defined in graph theory. Adopting the ONNX format would not work effectively and information would not be displayed properly (subviews' information being different from standard tensors').

the modularity needed to fully address KaNN's data type requirements

Can you provide an example or such a type or modularity need?

Although ONNX implementation does support many data types and could potentially meet our needs at one point, our KaNN implementation allows us to use any data type (whether as a single value or a list) at ANY point in the code without needing to specify individual compatibility for different parts (e.g. nodes, quantization...). This corresponds to our goal of improving our models in the near future to accept a wider variety of data types without having to modify Netron with each update.

Hope this helps you understand our need for a specific implementation of KaNN.