Open ivan-pi opened 1 year ago
Sorry for the late response--I didn't see this issue until now. Thanks for opening it. I didn't know about NNEF. This could remove the need for framework-specific adapters like that for Keras HDF5.
However, I'm worried about the level of implementation. I searched Keras and PyTorch docs for NNEF and each returned zero results. Is the support for writing NNEF files still in 3rd party libraries?
Related but different is ONNX.
In the meantime since this has been opened, I've heard and read a lot about ONNX and nothing about NNEF. I wonder if that means that ONNX is becoming mainstream or a de facto standard for model interexchange.
The NNEF page has added some text explaining the differences
ONNX and NNEF are Complementary ONNX moves quickly to track authoring framework updates NNEF provides a stable bridge from training into edge inferencing engines
NNEF | ONNX |
---|---|
Embedded Inferencing Import | Training Interchange |
Defined Specification | Open Source Project |
Multi-company Governance at Khronos | Initiated by Facebook & Microsoft |
Stability for hardware deployment | Software stack flexibility |
From what I understand, NNEF aims to be a stable format for (embedded?) inference engines (potentially with hardware support from vendors). I imagine it this way, a software update of the computer vision system in an autonomous car, might constitute simply replacing a NNEF file. The engineers training and tuning the models on the other hand, will exchange ONNX files. The NNEF page says you can export ONNX to NNEF and vice-versa.
So I guess ONNX would be the right one to support first.
The Khronos group has developed a standard data exchange format for neural networks: https://www.khronos.org/nnef. The latest specification is available here: https://registry.khronos.org/NNEF/specs/1.0/nnef-1.0.5.pdf. The purpose of the format is to provide a standard bridge between the various training frameworks and inference engines (e.g. train with Tensorflow, run inference in neural-fortran).
They also provide parsers in C++ and Python and a series of tests: https://github.com/KhronosGroup/NNEF-Tools/tree/master/parser
I'm guessing before committing to implementing and maintaining such a parser in Fortran, it would be cheaper to wrap the existing Khronos C++ parser in Fortran? This could also be brought to life as a module/library separate of neural-fortran.