search

allo- / virtual_webcam_background

Use a virtual webcam background and overlays with body-pix and v4l2loopback
GNU General Public License v3.0
307 stars 47 forks source link

Improve model download scripts #28

Open fangfufu opened 4 years ago

fangfufu commented 4 years ago

For mobilenet model with multiplier of 0.75, there are 2 shards:

How do I use that? Your script seems to only download/support one shard.

fangfufu commented 4 years ago

Ahh never mind, I figured it out...

fangfufu commented 4 years ago

I just want to say your stuff works really well, thanks! The only complaints I have is the model pulling scripts.

allo- commented 4 years ago

The full script is: https://github.com/ajaichemmanam/simple_bodypix_python/blob/master/get-model.sh

It does not have the best UX either ("Refer to https://storage.googleapis.com/tfjs-models for the available models" directs you to a long XML formatted file list), so my plan is to build something new in python.

My minimal script is from the time when I did not expose the model parameters in the config.

fangfufu commented 4 years ago

For anyone else who's reading this, to get multiplier 0.75, stride 16 code, these are the code:

#!/bin/sh
mkdir bodypix_mobilenet_float_075_model-stride16
wget -O bodypix_mobilenet_float_075_model-stride16/model.json https://storage.googleapis.com/tfjs-models/savedmodel/bodypix/mobilenet/float/075/model-stride16.json
wget -O bodypix_mobilenet_float_075_model-stride16/group1-shard1of2.bin https://storage.googleapis.com/tfjs-models/savedmodel/bodypix/mobilenet/float/075/group1-shard1of2.bin
wget -O bodypix_mobilenet_float_075_model-stride16/group1-shard2of2.bin https://storage.googleapis.com/tfjs-models/savedmodel/bodypix/mobilenet/float/075/group1-shard2of2.bin
allo- commented 4 years ago

I wonder why the source code refers to a model with multiplier=0.25 when it seems not to exist neither for mobilenet nor for resnet.

https://www.npmjs.com/package/body-pix-node does not mention it, but https://github.com/tensorflow/tfjs-models/blob/14aa081acf93d12ecf016e16c2c238e153de671d/body-pix/src/util.ts#L86 assigns the "low" option a multiplier of 0.25.

fangfufu commented 4 years ago

What does this multiplier thing even do?

allo- commented 4 years ago

Wait, I confused the two values.

Multiplier is a model parameter for mobilenet:

It is the float multiplier for the depth (number of channels) for all convolution ops. The larger the value, the larger the size of the layers, and more accurate the model at the cost of speed. A smaller value results in a smaller model and faster prediction time but lower accuracy.

-- https://github.com/tensorflow/tfjs-models/tree/master/body-pix/

internal_resolution is a scaling factor between 0.1 and 2.0 (I think you can choose other values, but 0.1-2.0 is reasonable) for the image. See

https://github.com/tensorflow/tfjs-models/blob/14aa081acf93d12ecf016e16c2c238e153de671d/body-pix/src/util.ts#L129