Run real time on less powerful devices?

vb000 / NeuriCam

Deep learning based video sensing method for low-power IoT cameras (Smart glasses, GoPro, Blink etc.).

https://arxiv.org/abs/2207.12496

MIT License

90 stars 9 forks source link

Run real time on less powerful devices? #1

Closed cTatu closed 2 years ago

cTatu commented 2 years ago

Hi,

In the paper you mentioned that the inference was made on an Nvidia RTX GPU and that the latency wasn't so great. Do you think that there could be some tweaks that could be done in order to execute inference in real time on a low power device like a smartphone or a laptop with a small iGPU?

vb000 commented 2 years ago

Hi @cTatu,

There are a few options that could be explored to run it in realtime, on a smaller form factor device:

Reducing the channel count from 64 to 48, 32 etc.
Using only 2-levels in the grid propagation instead of 4. With only 2-levels, the model could be considered as a bidirectional RNN for the most part, except for the cross connections between levels.
And of course, quantization (with/without retraining).

Accuracy trade-off required for these options has to be empirically evaluated. Hope this helps!

cTatu commented 2 years ago

Yes very helpful thank you very much!

cTatu commented 2 years ago

Hello again,

I have a question about spynet. It is necessary for inference as well? or just for training?

Thank you

vb000 commented 2 years ago

Hi,

It's required only for the training. If you do not wish to load spynet weights before inference, you change these two lines:

https://github.com/vb000/NeuriCam/blob/976a9a27403753e7fcd3f3aad404bd8d19a67ed8/model/keyvsrc/net.py#L34-L35

 spynet_pretrained=None,