search

JacobYuan7 / DIN-Group-Activity-Recognition-Benchmark

[ICCV 2021] A new codebase containing various methods for Group Activity Recognition. Paper title: Spatio-Temporal Dynamic Inference Network for Group Activity Recognition.
MIT License
53 stars 15 forks source link

Stage 2: FileNotFoundError basemodel_VD_res18.pth & basemodel_VD_vgg16.pth #2

Closed daniel-richter closed 2 years ago

daniel-richter commented 3 years ago

Thanks for sharing your code!

train_volleyball_stage1.py works on my machine, but any train_volleyball_stage2_* script throws FileNotFoundError - either result/basemodel_VD_res18.pth or result/basemodel_VD_vgg16.pth could not be found.

JacobYuan7 commented 3 years ago

@daniel-richter These two files are the pre-trained base models. I can release a link for the pre-trained models I use if you need it.

daniel-richter commented 3 years ago

As far as I understand I could not execute any train_volleyball_stage2_* script without those files, right? Or will train_volleyball_stage1.py create these files after finishing?

JacobYuan7 commented 3 years ago

As far as I understand I could not execute any train_volleyball_stage2_* script without those files, right? Or will train_volleyball_stage1.py create these files after finishing?

  1. A better choice is to run "train_volleyballstage2*" with the pretraining model. It should produce better results.
  2. Yes, just choose the model with the best performance after running train_volleyball_stage1.py.
daniel-richter commented 3 years ago

Currently, I can't run "train_volleyballstage2*". There is either a FileNotFoundError for "result/basemodel_VD_res18.pth" or "result/basemodel_VD_vgg16.pth". Where can I find these files? Or how can I generate them?

A better choice is to run "train_volleyballstage2*" with the pretraining model. It should produce better results.

Where can I find this "pretraining model" you mentioned?

Yes, just choose the model with the best performance after running train_volleyball_stage1.py.

But what do I have to do with this file? I habe a folder result/[Volleyball_stage1_stage1]<2021-09-23_12-46-16> with the best result stored in stage1_epoch192_86.39%.pth. And now? How can I use this file? Do I habe to rename and/or move it to some specific location?

JacobYuan7 commented 3 years ago

Currently, I can't run "train_volleyballstage2*". There is either a FileNotFoundError for "result/basemodel_VD_res18.pth" or "result/basemodel_VD_vgg16.pth". Where can I find these files? Or how can I generate them?

A better choice is to run "train_volleyballstage2*" with the pretraining model. It should produce better results.

Where can I find this "pretraining model" you mentioned?

Yes, just choose the model with the best performance after running train_volleyball_stage1.py.

But what do I have to do with this file? I habe a folder result/[Volleyball_stage1_stage1]<2021-09-23_12-46-16> with the best result stored in stage1_epoch192_86.39%.pth. And now? How can I use this file? Do I habe to rename and/or move it to some specific location?

Yes, you should rename it and move it to the specific file path like 'result/basemodel_VD_res18.pth'. BTW, why is the performance of your base model lower than mine? Do you change any setting?

daniel-richter commented 3 years ago

Hm, okay. The thing is, that there isn't anything mentioned in the docs how to get from stage 1 to stage 2.

In train_volleyball_stage2_arg, train_volleyball_stage2_at.py, train_volleyball_stage2_higcin.py, train_volleyball_stage2_pctdm.py and train_volleyball_stage2_sacrf_biute.py, the part vgg16 setup is disabled while res18 setup is enabled. https://github.com/JacobYuan7/DIN_GAR/blob/7c5a8f54cc3b713ead438cc47dd54c491c837fbf/scripts/train_volleyball_stage2_arg.py#L17-L27

By contrast, in train_volleyball_stage2_dynamic_tce.py and train_volleyball_stage2_dynamic.py res18 setup is disabled while vgg16 setup is enabled. https://github.com/JacobYuan7/DIN_GAR/blob/7c5a8f54cc3b713ead438cc47dd54c491c837fbf/scripts/train_volleyball_stage2_dynamic.py#L16-L26

Yes, you should rename it and move it to the specific file path like 'result/basemodel_VD_res18.pth'.

In train_volleyball_stage1.py there are the following lines. I assume that this exact code is used to (only) create the file result/basemodel_VD_vgg16.pth https://github.com/JacobYuan7/DIN_GAR/blob/7c5a8f54cc3b713ead438cc47dd54c491c837fbf/scripts/train_volleyball_stage1.py#L14-L18

So the questions are:

BTW, why is the performance of your base model lower than mine? Do you change any setting?

I didn't change any source file. I executed your scripts inside the Docker container I pushed in https://github.com/JacobYuan7/DIN_GAR/pull/4. It's Ubuntu 16.04.6 LTS, CUDA 10.0.130, cuDNN 7.6.03, Python 3.6.9, PyTorch 1.2.0, TorchVision 0.4.0

JacobYuan7 commented 3 years ago

@daniel-richter

Hm, okay. The thing is, that there isn't anything mentioned in the docs how to get from stage 1 to stage 2.

Actually, it's mentioned in the implementation details of the paper. You can check it out.

Do I have to modify train_volleyball_stage1.py to create the result/basemodel_VD_res18.pth?

If you are using vgg16 base model, then you should comment out these lines https://github.com/JacobYuan7/DIN_GAR/blob/7c5a8f54cc3b713ead438cc47dd54c491c837fbf/scripts/train_volleyball_stage2_dynamic.py#L23-L26 and use these lines https://github.com/JacobYuan7/DIN_GAR/blob/7c5a8f54cc3b713ead438cc47dd54c491c837fbf/scripts/train_volleyball_stage2_dynamic.py#L17-L20 Similar things can be conducted for the res18 setup. Other script files are similar to this one. I did not deliberately enable res18 setup in other files. You can enable whichever setup you prefer as long as you prepare the right base model.

Is it possible to switch from res18 to vgg16 (and vice-versa) in train_volleyballstage2*?

Of course, it is possible as long as you prepare the right base model.

daniel-richter commented 3 years ago

Thanks for the clarification. I ran train_volleyball_stage1.py and renamed the best model to basemodel_VD_vgg16.pth. Then I changed the cfg.backbone part of the train_volleyball_stage2_* scripts what seems to work.

You can enable whichever setup you prefer as long as you prepare the right base model.

What do I have to do to create the basemodel_VD_res18.pth file? If I change the cfg.backbone part in train_volleyball_stage1.py to res18, an Assert false occurs. https://github.com/JacobYuan7/DIN_GAR/blob/7c5a8f54cc3b713ead438cc47dd54c491c837fbf/scripts/train_volleyball_stage1.py#L15 The reason is that the use of res18 is apparently not intended there: https://github.com/JacobYuan7/DIN_GAR/blob/7c5a8f54cc3b713ead438cc47dd54c491c837fbf/base_model.py#L19-L26

JacobYuan7 commented 2 years ago

@daniel-richter Sorry, I add res18 support to the base_model training. Thanks for your careful check.