search

attractivechaos / kann

A lightweight C library for artificial neural networks
Other
678 stars 117 forks source link

Format of model file #11

Open mostafaelhoushi opened 6 years ago

mostafaelhoushi commented 6 years ago

I would like to use a model that is pre-trained on Keras or Tensorflow, and run it on kann. I am trying to look for the file format that the weights needs to be saved in to load it from with kann. Please advise.

g2cacheQ commented 5 years ago

I have similar question - ie what would be involved in importing a pre-trained network via the open source ONNX format?

https://github.com/onnx/models/tree/master/mnist

attractivechaos commented 5 years ago

No. ONNX is based on protobuf, which alone is many times larger than KANN. Supporting ONNX would defeat the goal of this project.

g2cacheQ commented 5 years ago

I'm not sure I follow how ONNX would defeat the goal of the kann project. Protocol buffers are merely binary serialization format - the size and complexity directly proportional to the object model you are serializing. The purpose of ONNX is to provide an open mechanism for sharing trained network data between frameworks. People will not realistically write custom training routines with kann. I see ONNX as a translation layer - possibly a 1-way street. I like the lightweight model implementation, but am not interested in re-engineering the creation of established models, then training them. I would like to deploy a trained model with kann and use a "real" framework for training. I apologize if my use of the term "real" seems derogatory - it's not intended to be so. Just that Caffe, Tensorflow, Keras etc are established training frameworks, and I don't see kann as a viable competitor. Regardless, it would be helpful to describe how one applies/updates the weights to a kann model. Thank you for your work, this is very interesting.

attractivechaos commented 5 years ago

Parsing protobuf requires a protobuf-c, which is about the same size as kann. You additionally need translation code. With ONNX, kann won't be lightweight any more. ONNX support should be a separate project, not part of kann itself.

In fact, if the goal is to deploy the model, the much cleaner way is to start a brand-new project, taking bits of kann code and fit them into structs matching the ONNX definitions. The majority of code in kann is for autodiff and training. The forward pass can be implemented in ~1000 lines. Creating a translation layer between kann and ONNX probably needs more than that amount of code.

Anyway, I buy your point, but that is not happening in this repo.