search

modern-fortran / neural-fortran

A parallel framework for deep learning
MIT License
393 stars 82 forks source link

Support for neural network exchange format (NNEF) #102

Open ivan-pi opened 1 year ago

ivan-pi commented 1 year ago

The Khronos group has developed a standard data exchange format for neural networks: https://www.khronos.org/nnef. The latest specification is available here: https://registry.khronos.org/NNEF/specs/1.0/nnef-1.0.5.pdf. The purpose of the format is to provide a standard bridge between the various training frameworks and inference engines (e.g. train with Tensorflow, run inference in neural-fortran).

They also provide parsers in C++ and Python and a series of tests: https://github.com/KhronosGroup/NNEF-Tools/tree/master/parser

I'm guessing before committing to implementing and maintaining such a parser in Fortran, it would be cheaper to wrap the existing Khronos C++ parser in Fortran? This could also be brought to life as a module/library separate of neural-fortran.

milancurcic commented 1 year ago

Sorry for the late response--I didn't see this issue until now. Thanks for opening it. I didn't know about NNEF. This could remove the need for framework-specific adapters like that for Keras HDF5.

However, I'm worried about the level of implementation. I searched Keras and PyTorch docs for NNEF and each returned zero results. Is the support for writing NNEF files still in 3rd party libraries?

Related but different is ONNX.

milancurcic commented 1 year ago

In the meantime since this has been opened, I've heard and read a lot about ONNX and nothing about NNEF. I wonder if that means that ONNX is becoming mainstream or a de facto standard for model interexchange.

ivan-pi commented 1 year ago

The NNEF page has added some text explaining the differences

ONNX and NNEF are Complementary ONNX moves quickly to track authoring framework updates NNEF provides a stable bridge from training into edge inferencing engines

NNEF ONNX
Embedded Inferencing Import Training Interchange
Defined Specification Open Source Project
Multi-company Governance at Khronos Initiated by Facebook & Microsoft
Stability for hardware deployment Software stack flexibility

From what I understand, NNEF aims to be a stable format for (embedded?) inference engines (potentially with hardware support from vendors). I imagine it this way, a software update of the computer vision system in an autonomous car, might constitute simply replacing a NNEF file. The engineers training and tuning the models on the other hand, will exchange ONNX files. The NNEF page says you can export ONNX to NNEF and vice-versa.

So I guess ONNX would be the right one to support first.