facebookresearch / VMZ

VMZ: Model Zoo for Video Modeling
Apache License 2.0
1.04k stars 155 forks source link

Training .mdl to Predictor #18

Open mkummer225 opened 6 years ago

mkummer225 commented 6 years ago

Hey team,

Thanks for all your awesome work in Spatio-Temporal CNNs! I had a couple quick questions about going from a trained model (.mdl format) to a standalone predictor.

I was able to successfully build the environment and train the R(2+1)D model on my own dataset but now that I've got my trained .mdl file I've been struggling to find documentation on how to go from the trained/training model that takes in LMDB's to a 'predictor' model that can take in plain np array inputs.

I've currently got my .mdl loaded into an init_net and predict_net as suggested by much of the documentation I was able to find:

init_net proto:

input: "!!PREDICTOR_DBREADER"
output: "gpu_0/conv1_middle_w"
output: "gpu_0/conv1_middle_spatbn_relu_s"
output: "gpu_0/conv1_middle_spatbn_relu_b"
output: "gpu_0/conv1_w"
output: "gpu_0/conv1_spatbn_relu_s"
output: "gpu_0/conv1_spatbn_relu_b"
output: "gpu_0/comp_0_conv_1_middle_w"
output: "gpu_0/comp_0_spatbn_1_middle_s"
...
name: ""
type: "Load"

first two layers in predict_net proto:

input: "r2plus1d_train_init/CreateDB"
output: "gpu_0/data"
output: "gpu_0/label"
name: "data"
type: "VideoInput"
arg {
... bunch of VideoInput args ...
}
device_option {
  device_type: 1
  cuda_gpu_id: 0
}
input: "gpu_0/data"
output: "gpu_0/data"
name: ""
type: "StopGradient"
device_option {
  device_type: 1
  cuda_gpu_id: 0
}
{convolutional layers, etc...}

My questions are: a) how would I go about modifying the net(s) to allow for individual predictions from raw np arrays? Ex:

data = np.random.rand(N, C, H, W) // aka np arrays of video frames
workspace.FeedBlob("gpu0_data", data)

and b) how would I switch from GPU to CPU execution? Is there something like workspace.RunAllOnCPU() for example?

Thanks!