facebookresearch / video-long-term-feature-banks

Long-Term Feature Banks for Detailed Video Understanding
Apache License 2.0
373 stars 62 forks source link

Path setting at the training step2 #62

Open LeeJunHyun opened 3 years ago

LeeJunHyun commented 3 years ago

Hi, I really appreciate your commitment for this remarkable work.

I have a little difficulty to set up LFB path at training.

According to GETTING_STARTED.md, there are two steps for training the model over Charades dataset.

Stage 1:

python2 tools/train_net.py \
  --config_file configs/charades_r50_baseline.yaml \
  CHECKPOINT.DIR /tmp/lfb-nl-stage1-output

Stage 2:

python2 tools/train_net.py \
  --config_file configs/charades_r50_lfb_nl.yaml \
  TRAIN.PARAMS_FILE [path to model weight from Stage 1] \
  LFB.MODEL_PARAMS_FILE [path to model weight for LFB inference] \
  LFB.WRITE_LFB True \
  CHECKPOINT.DIR /tmp/lfb-nl-stage2-output

After Stage 1, I got some files under CHECKPOINT.DIR (/tmp/lfb-nl-stage1-output)

├── checkpoints
│   ├── c2_model_iter12000.pkl
│   ├── c2_model_iter16000.pkl
│   ├── c2_model_iter24000.pkl
│   ├── c2_model_iter4000.pkl
│   ├── c2_model_iter8000.pkl
│   └── converted_model.pkl
├── CNN_1.pbtxt
├── CNN_init_1.pbtxt
├── CNN_init.pbtxt
└── CNN.pbtxt

In here, which path can be proper for TRAIN.PARAMS_FILE [path to model weight from Stage 1] and LFB.MODEL_PARAMS_FILE [path to model weight for LFB inference] at Stage 2? I thought c2_model_iter24000.pkl can be assigned as TRAIN.PARAMS_FILE, but there was an error (KeyError: u'nonlocal_conv3_1_bn_rm' at remove_spatial_bn_layers(c2cls_weights)) and not sure about LFB.MODEL_PARAMS_FILE. Could you please give me some specific example filenames for both *PARAMS_FILEs?

Thanks, Junhyun Lee