search

larq / compute-engine

Highly optimized inference engine for Binarized Neural Networks
https://docs.larq.dev/compute-engine
Apache License 2.0
243 stars 35 forks source link

investigating TF lite SELECTIVE_REGISTRATION feature #76

Open arashb opened 5 years ago

arashb commented 5 years ago

the feature SELECTIVE_REGISTRATION in TF Lite allows to create smaller TF lite library by only including the ops used in the converting model (see here). I couldn't find proper doc in the official TF lite guide though.

Tombana commented 5 years ago

So from what I understand, it works something like this:

There is the register.cc file which registers all the kernels. The way that we add our own kernel is by providing register_lqce.cc which includes our kernels. Now the nice thing about a linker, is that when you do not register your kernel there, it will automatically detect that your Register_MyKernel() function is never called and it will remove that function from your final binary. So even if you compiled your MyKernel.cc file, it will just be removed at the end because it is seen as "unused".

So basically when you enable the selective registration feature, it generates a new register.cc file for you and it only calls the Register_XXX functions for those ops that are in your model. This way you get a binary that can only do your specific model or a subset of it.

So by providing our own register.cc file we are already kind-of using a system similar to this.