Tensorflow or Tensorflow Lite model of Silero VAD

[Nikaa02]

🚀 Feature

Publish open source Silero VAD model on TensorFlow or TensorFlow Lite

Motivation

I wish to use your Silero VAD model in a production environment where only TF is supported

Pitch

Silero VAD would be very useful on mobile and embedded devices. TensorFlow Lite is the best variant in context of devices with limited memory and capacity.

Alternatives

I tried to convert both published models to TF but faced with different problems, maybe because of mutable input size or torchscript and onnxruntime instead of classical torch and onnx types.

Additional context

Thank you in advance.

[snakers4]

Hi,

Many thanks for your interest. This is an interesting issue.

Note that PyTorch now has mobile and it also light binaries for mobile released in 1.8 (70% size reduction).

ONNX is also an option for edge deployment - people were able to launch it on ARM64.

As for the TF:

• We have investigated the standard ONNX-TF exporter for silero-models and it kind of works. But the tf-lite model exporter does not in typical Google fashion with cryptic errors. While I can argue that the problem is with TF, which always has been that way, most likely it is with triple conversion (torch > onnx > tf);

• It is also an issue the way TF treats its community and TF hub. To put it simply - there is zero traffic from TF hub and the way it is managed Google essentially tries not to give you credit at all, i.e. they rehost your files / do not link your repo / etc. Basically torch hub is for promoting people's work and TF hub (and other hubs - google added rehosting after I wrote this - basically they try to mimic the hugging face model where people take credit for others work);

• For silero-models we received many requests from people willing to use your models commercially with TF (and on other platforms) disregarding the license. When taking the situation into account we provided commercial quotes for work - they vanished;

[snakers4]

So to sum this up:

• To do tflite conversion properly you need to reimplement the models from scratch in TF and manually load the weights and then convert - we are not willing to do this for free and there is little benefit / value for us. You are encouraged to use PyTorch or ONNX. If you would like to proceed with tf-lite - please contact me directly, I will provide a commercial quote for this work;

• I believe we will drop support for tf models and tf hub in next version of silero-midels;

[snakers4]

Some weekly stats just for the sake of argument