How to load pretranid 3D model?

[fengwhu]

@chuckcho I want to load the pre-trained Sport1m model via net = caffe.Classifier(MODEL_FILE, PRETRAINED) in video-caffe. I used the prototxt file provided by you in #https://github.com/facebook/C3D/pull/59, but I failed to load the 3D model conv3d_deepnetA_sport1m_iter_1900000. I wonder how to modify the prototxt file if using function net = caffe.Classifier() in video-caffe?

[chuckcho]

It doesn't look like a trivial task to use a model generated by different Caffe (facebook/C3D) because there have been fundamental changes in blob definitions, and etc. If I have a good Sport1m or UCF101 model trained using video-caffe later, I'll share it.

[fengwhu]

@chuckcho Thank you!

[fengwhu]

@chuckcho Excuse me. I wonder if the good Sport1m or UCF101 model is available for video-caffe. Could you share it?

[chuckcho]

Yes. Will share it when I have a good one. Thanks.

On Thu, Mar 24, 2016 at 8:55 AM, fengwhu notifications@github.com wrote:

@chuckcho https://github.com/chuckcho Excuse me. I wonder if the good Sport1m or UCF101 model is available for video-caffe. Could you share it?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/chuckcho/video-caffe/issues/3#issuecomment-200820116

Chuck Cho Dextro, chuck.cho@dextro.co

[attitudechunfeng]

Hi, all. I have transplanted a model from the original conv3d_deepnetA_sports1m_1900000 model. I tried to make a fork from the master, but when i tried to upload the model file, it reminds me the file is too big so i can't do it. How could i deal with this problem? @chuckcho.

[attitudechunfeng]

I don't check its correctness, but I've uploaded the model file to onedrive and you guys can get it from the link below: https://onedrive.live.com/redir?resid=EEBF43C91B5A42A8!1524&authkey=!AJmM9Jmv4h6uDJA&ithint=file%2ccaffemodel its 'deploy.prototxt' file is as follows:

name: "c3d_ucf101"

# ----- video/label input -----
input: "data"
input_shape {
  dim: 10
  dim: 3
  dim: 16
  dim: 112
  dim: 112
}

# ----- 1st group -----
layer {
  name: "conv1a"
  type: "NdConvolution"
  bottom: "data"
  top: "conv1a"
  convolution_param {
    num_output: 64
    kernel_shape { dim: 3 dim: 3 dim: 3 }
    stride_shape { dim: 1 dim: 1 dim: 1 }
    pad_shape    { dim: 1 dim: 1 dim: 1 }
  }
}
layer {
  name: "relu1a"
  type: "ReLU"
  bottom: "conv1a"
  top: "conv1a"
}
layer {
  name: "pool1"
  type: "NdPooling"
  bottom: "conv1a"
  top: "pool1"
  pooling_param {
    pool: MAX
    kernel_shape { dim: 1 dim: 2 dim: 2 }
    stride_shape { dim: 1 dim: 2 dim: 2 }
  }
}

# ----- 2nd group -----
layer {
  name: "conv2a"
  type: "NdConvolution"
  bottom: "pool1"
  top: "conv2a"
  convolution_param {
    num_output: 128
    kernel_shape { dim: 3 dim: 3 dim: 3 }
    stride_shape { dim: 1 dim: 1 dim: 1 }
    pad_shape    { dim: 1 dim: 1 dim: 1 }
  }
}
layer {
  name: "relu2a"
  type: "ReLU"
  bottom: "conv2a"
  top: "conv2a"
}
layer {
  name: "pool2"
  type: "NdPooling"
  bottom: "conv2a"
  top: "pool2"
  pooling_param {
    pool: MAX
    kernel_shape { dim: 2 dim: 2 dim: 2 }
    stride_shape { dim: 2 dim: 2 dim: 2 }
  }
}

# ----- 3rd group -----
layer {
  name: "conv3a"
  type: "NdConvolution"
  bottom: "pool2"
  top: "conv3a"
  convolution_param {
    num_output: 256
    kernel_shape { dim: 3 dim: 3 dim: 3 }
    stride_shape { dim: 1 dim: 1 dim: 1 }
    pad_shape    { dim: 1 dim: 1 dim: 1 }
  }
}
layer {
  name: "relu3a"
  type: "ReLU"
  bottom: "conv3a"
  top: "conv3a"
}
layer {
  name: "conv3b"
  type: "NdConvolution"
  bottom: "conv3a"
  top: "conv3b"
  convolution_param {
    num_output: 256
    kernel_shape { dim: 3 dim: 3 dim: 3 }
    stride_shape { dim: 1 dim: 1 dim: 1 }
    pad_shape    { dim: 1 dim: 1 dim: 1 }
  }
}
layer {
  name: "relu3b"
  type: "ReLU"
  bottom: "conv3b"
  top: "conv3b"
}
layer {
  name: "pool3"
  type: "NdPooling"
  bottom: "conv3b"
  top: "pool3"
  pooling_param {
    pool: MAX
    kernel_shape { dim: 2 dim: 2 dim: 2 }
    stride_shape { dim: 2 dim: 2 dim: 2 }
  }
}

# ----- 4th group -----
layer {
  name: "conv4a"
  type: "NdConvolution"
  bottom: "pool3"
  top: "conv4a"
  convolution_param {
    num_output: 512
    kernel_shape { dim: 3 dim: 3 dim: 3 }
    stride_shape { dim: 1 dim: 1 dim: 1 }
    pad_shape    { dim: 1 dim: 1 dim: 1 }
  }
}
layer {
  name: "relu4a"
  type: "ReLU"
  bottom: "conv4a"
  top: "conv4a"
}
layer {
  name: "conv4b"
  type: "NdConvolution"
  bottom: "conv4a"
  top: "conv4b"
  convolution_param {
    num_output: 512
    kernel_shape { dim: 3 dim: 3 dim: 3 }
    stride_shape { dim: 1 dim: 1 dim: 1 }
    pad_shape    { dim: 1 dim: 1 dim: 1 }
  }
}
layer {
  name: "relu4b"
  type: "ReLU"
  bottom: "conv4b"
  top: "conv4b"
}
layer {
  name: "pool4"
  type: "NdPooling"
  bottom: "conv4b"
  top: "pool4"
  pooling_param {
    pool: MAX
    kernel_shape { dim: 2 dim: 2 dim: 2 }
    stride_shape { dim: 2 dim: 2 dim: 2 }
  }
}

# ----- 5th group -----
layer {
  name: "conv5a"
  type: "NdConvolution"
  bottom: "pool4"
  top: "conv5a"
  convolution_param {
    num_output: 512
    kernel_shape { dim: 3 dim: 3 dim: 3 }
    stride_shape { dim: 1 dim: 1 dim: 1 }
    pad_shape    { dim: 1 dim: 1 dim: 1 }
  }
}
layer {
  name: "relu5a"
  type: "ReLU"
  bottom: "conv5a"
  top: "conv5a"
}
layer {
  name: "conv5b"
  type: "NdConvolution"
  bottom: "conv5a"
  top: "conv5b"
  convolution_param {
    num_output: 512
    kernel_shape { dim: 3 dim: 3 dim: 3 }
    stride_shape { dim: 1 dim: 1 dim: 1 }
    pad_shape    { dim: 1 dim: 1 dim: 1 }
  }
}
layer {
  name: "relu5b"
  type: "ReLU"
  bottom: "conv5b"
  top: "conv5b"
}
layer {
  name: "pool5"
  type: "NdPooling"
  bottom: "conv5b"
  top: "pool5"
  pooling_param {
    pool: MAX
    kernel_shape { dim: 2 dim: 2 dim: 2 }
    stride_shape { dim: 2 dim: 2 dim: 2 }
  }
}

# ----- 1st fc group -----
layer {
  name: "fc6"
  type: "InnerProduct"
  bottom: "pool5"
  top: "fc6"
  inner_product_param {
    num_output: 4096
  }
}
layer {
  name: "relu6"
  type: "ReLU"
  bottom: "fc6"
  top: "fc6"
}
layer {
  name: "drop6"
  type: "Dropout"
  bottom: "fc6"
  top: "fc6"
  dropout_param {
    dropout_ratio: 0.5
  }
}

# ----- 2nd fc group -----
layer {
  name: "fc7"
  type: "InnerProduct"
  bottom: "fc6"
  top: "fc7"
  inner_product_param {
    num_output: 4096
  }
}
layer {
  name: "relu7"
  type: "ReLU"
  bottom: "fc7"
  top: "fc7"
}
layer {
  name: "drop7"
  type: "Dropout"
  bottom: "fc7"
  top: "fc7"
  dropout_param {
    dropout_ratio: 0.5
  }
}

# ----- 3rd fc group -----
layer {
  name: "fc8"
  type: "InnerProduct"
  bottom: "fc7"
  top: "fc8"
  inner_product_param {
    num_output: 1
  }
}

Just enjoy it.

[chuckcho]

@attitudechunfeng, awesome! will try this out. thanks for your contribution.

[fengwhu]

@attitudechunfeng Sorry. I still can't load the transplanted Sport1m model with its 'deploy.prototxt' file by net = caffe.Classifier(MODEL_FILE, PRETRAINED) in video-caffe. The python console gets error: It seems the kernel died unexpectedly. Use 'Restart kernel' to continue using this console.

[attitudechunfeng]

@fengwhu I've not tried python interface before, however, when i use command interface it works well. Maybe you can try the command way.

[fengwhu]

@attitudechunfeng Could you share your command interface codes for this?

[attitudechunfeng]

@fengwhu for example, if you'd like to fine-tune the pre-trained model. First, u need to write a "train_test.prototxt" like the "deploy.prototxt", also with a "solver.prototxt". Make sure all the path in above files right. Then run the command ../../build/tools/caffe train --solver=solver.prototxt --weights=../../conv3d_deepnetA_sport1m_iter_1900000.caffemodel u can make a try.

[fengwhu]

@attitudechunfeng I write a "train_test.prototxt" with a "solver.prototxt" and run the command just as you said. But I am getting the following error when using conv3d_deepnetA_sport1m_iter_1900000.caffemodel

I0416 17:42:56.640152 13253 caffe.cpp:129] Finetuning from conv3d_deepnetA_sport1m_iter_1900000.caffemodel
F0416 17:42:56.640224 13253 upgrade_proto.cpp:75] Check failed: ReadProtoFromBinaryFile(param_file, param) Failed to parse NetParameter file: conv3d_deepnetA_sport1m_iter_1900000.caffemodel
*** Check failure stack trace: ***
    @     0x7feebd52c9fd  google::LogMessage::Fail()
    @     0x7feebd52e89d  google::LogMessage::SendToLog()
    @     0x7feebd52c5ec  google::LogMessage::Flush()
    @     0x7feebd52f1be  google::LogMessageFatal::~LogMessageFatal()
    @     0x7feebdc16c3e  caffe::ReadNetParamsFromBinaryFileOrDie()
    @     0x7feebdab96a7  caffe::Net<>::CopyTrainedLayersFromBinaryProto()
    @     0x7feebdab9716  caffe::Net<>::CopyTrainedLayersFrom()
    @           0x409284  CopyLayers()
    @           0x40ae45  train()
    @           0x4085fc  main
    @     0x7feebc254ec5  (unknown)
    @           0x408f32  (unknown)
Aborted (core dumped)

[haohao900618]

@fengwhu Hello, I encountered the same problem like you. Did you have solved it. If you have solved the problem, please share your experience here! Thank you very much!

[fengwhu]

@haohao900618 Sorry, I haven't solved the problem yet.

[mzolfaghari]

@attitudechunfeng Is it possible to provide your train.prototxt? Because I get same error as @haohao900618 and @fengwhu. Also, why did you set number of outputs in your deploy.prototx to 1? ( num_output: 1)

Thanks,

[mzolfaghari]

@chuckcho Could you please share your trained models? specially, trained models on sport1m dataset.

Best,

[chuckcho]

unfortunately i don't have trained models on Sports1m.

[mzolfaghari]

@chuckcho , @fengwhu , @haohao900618 : Maybe this could be interesting for you: https://gist.github.com/bearpaw/5c38196a3f0c755621b8 converting new caffemodel to old caffe!

Best,

[chuckcho]

@mzolfaghari cool. thx.

[chuckcho]

For those who follow this thread, I've posted a sport1m-pretrained model. Look here for the detail: https://github.com/chuckcho/video-caffe/issues/21#issuecomment-228075641

[icbcbicc]

@attitudechunfeng Sorry to bother, I want to use a model which I have already trained to classify some videos. I follow your suggestions but I still encounter some problems. Is the train_test.prototxt" same with the "deploy.prototxt"? What's more, can you share your solver.prototxt? Thanks.

[icbcbicc]

@chuckcho In order to classify some videos after training, I use ./build/examples/cpp_classification/classification.bin together with the deploy.prototxt given above, but it seems that it only supports image input. What should I do to get the labels? Thanks.

[chuckcho]

@icbcbicc i don't have a python script for that, but you can prepare test video as in UCF-101 example, and use test script.

[icbcbicc]

@chuckcho I have tried this test script before, but it only outputs the accuracy of each iteration rather than the labels.

[pelun]

Everyone can help me step-by-step what I can do after I download source code "video-caffe". I want to run this project. When I run extract_UCF-101_frames.sh, I modified directory but it show error syntax in /. Thanks