Train with custom dataset

[Ai-is-light]

@rocksyne Thanks for your link and materials, after using the sports1m_finetuning_ucf101.model into the folder, when l train the network, I got the error as following: InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [27] rhs shape= [101] [[Node: save/Assign_10 = Assign[T=DT_FLOAT, _class=["loc:@var_name/bout"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](var_name/bout, save/RestoreV2:10)]]

and, my custom dataset's categories are 27, the original categories are 101, I can't use the original pretrained model to train the new model? I'm counfused> @rocksyne @hx173149 Looking forward to any replies

[rocksyne]

I am sorry for my late response.

There is a fundamental you are missing. The classifier for this model is of 101 classes. If you want to use it for 27 action classes, you will have to use the concept of fine tuning or transfer learning (the terms are used inter changeably).

With this, you can remove the top 101 classifier and replace it with a 27 class classifier. If you can share a summary of your code, I may be able to help.

[953585130]

@hx173149 @rocksyne @chuckcho Hi， I have already achieved better precision on UCF101 according to the tutorial, but I now want to use my own video data to fine tune with the provided pre-training model (I have a very big doubt: train_c3d_ucf101.py in the tensorflow folder actually That is, fine-tuning the code, can you modify the pre-training model you need in the code?). My video data has 20 categories. Do I need to modify softmax to correspond to my own number of categories? Looking forward to your reply! Thank you!

[rocksyne]

@953585130 First, be sure that the pre trained model did not use UCF101 a the data-set. If that is the case, you are bound to have very good accuracy (95 ~100%) and this is not good. This is because you are just showing it data that it has already seen.

If you are however using the pre trained model based on SPORTS1M data-set, then you are on the right path. That one has about 487 classes. You can remove that last layer and replace it with your 20 class classifier. @hx173149 has done a very good job at this work but I am sure with a little effort, you can convert the codes to Keras and have some more ease. This way you will be able to remove and add layers with ease for your fine tuning process.

I am not sure if I have answered you correctly, but please do let me know if you need some specific help.

Cheers!

[953585130]

@rocksyne Thank you very much for your reply! I have just been exposed to deep learning, and there are indeed many problems. I reproduced the experiment using the sportm_finetuning_ucf101.model model provided by UCF and the author, and the effect was good. And now I want to use this tensorflow-based C3D to try the HMDB51 dataset, this dataset has only 51 classes.

The data is pre-processed, and I modified it in the c3d_model.py file (NUM_CLASSES=51). An error occurred while executing the train_c3d.py script, because the original pre-training model (sportm_finetuning_ucf101.model) was used during training. The category is different and an error has occurred.

Create model directory

if not os.path.exists(model_save_dir): os.makedirs(model_save_dir)a use_pretrained_model = True model_filename = "./sports1m_finetuning_ucf101.model" When I change true to false, the code will run normally. My doubt is that if you use someone else's pre-training model when you train your own dataset, is this process a fine-tuning process?

Without the pre-training model, I trained the HMDB51 dataset from scratch. The accuracy of the validation set was only about 40%, so I think I need a pre-training model (large benchmark model), but I don't know the choice. Can that match the current data set?

[rocksyne]

@953585130 I am glad to help! So here is the thing. When you take a pre trained model, like sportm_finetuning_ucf101.model, the last layer already has the number of classes as classifier. In this case, it is 101. Therefore, if you change this number to anything rather than 101, you will run into dimensional problems and some other strange ones I do not remember on top of my head.

However, if you train from scratch, you will be using a the C3D architecture with your own number of classes (classifier) at the last layer, and all things being equal, this should run very nicely. Of course you will get some not very encouraging results until you have tuned a number of the hyper-parameters

Now back to sportm_finetuning_ucf101.model. Uing this model will help you get better results. If you want to use this mode, this is what you must do.

• Remove the last layer, that is, the classifier
• Add your classifier, that is, a dense layer with your specified number of classes.

You will not run into problems if you do this. I am still working on properly documenting the keras version of this to help others who need it. @hx173149 has done a great job but unfortunately, I am not proficient in Tensorflow so it is a little hard for me to resolve issues that come up in that domain.

I have tried this in keras and I must say the results were encouraging. If you are conversant with Keras, I will advise you give it a try. Tensoflow is for people who have a lot of time to spare. But if you want to do quick prototyping, Keras will make your life several times easier.

Cheers!

[Malathi15]

Hi @rocksyne , I would like to train this model using UCF-crime dataset. What are the changes I have to make?

[rocksyne]

@Malathi15 Could you please provide a little more detail?

[MathCau]

@rocksyne example to freeze only the forst conv layer 👍 if os.path.isfile(model_filename) and use_pretrained_model:

  var_list = [v for v in tf.trainable_variables() if v.name.find("c1") != -1]
  saver = tf.train.Saver(var_list)
  saver.restore(sess, model_filename)

[1774467982]

@Malathi15 Could you please provide a little more detail?

@953585130 I am glad to help! So here is the thing. When you take a pre trained model, like sportm_finetuning_ucf101.model, the last layer already has the number of classes as classifier. In this case, it is 101. Therefore, if you change this number to anything rather than 101, you will run into dimensional problems and some other strange ones I do not remember on top of my head.

However, if you train from scratch, you will be using a the C3D architecture with your own number of classes (classifier) at the last layer, and all things being equal, this should run very nicely. Of course you will get some not very encouraging results until you have tuned a number of the hyper-parameters

Now back to sportm_finetuning_ucf101.model. Uing this model will help you get better results. If you want to use this mode, this is what you must do.

• Remove the last layer, that is, the classifier
• Add your classifier, that is, a dense layer with your specified number of classes.

You will not run into problems if you do this. I am still working on properly documenting the keras version of this to help others who need it. @hx173149 has done a great job but unfortunately, I am not proficient in Tensorflow so it is a little hard for me to resolve issues that come up in that domain.

I have tried this in keras and I must say the results were encouraging. If you are conversant with Keras, I will advise you give it a try. Tensoflow is for people who have a lot of time to spare. But if you want to do quick prototyping, Keras will make your life several times easier.

Cheers!

@Malathi15 Could you please provide a little more detail?

I really need your help，when i trained C3D，error happened “list index out of range”，i can not slove it,can you help me?