SlowFast をfine-tuning

idekazuki commented 4 years ago

https://github.com/facebookresearch/SlowFast/blob/6ff9dd9cdd/slowfast/config/defaults.py#L171 モデルのクラス数を変更するにはこのMODEL.NUM_CLASSESを変更すれば良さそう。

ここで気づいたが、今までKinetics600のデータセットを試していたので精度が０なのではないか。横道にそれるが、kinetics-400のcsvデータから動画セットを作成し、検証してみる。

[INFO: logging.py: 67]: json_stats: {"split": "test_final", "top1_acc": "0.00", "top5_acc": "9.09"}

top5_acc が9.09になっている。精度は気になるが、成功した。きちんと動いている。

idekazuki commented 4 years ago

試しに以下のようなコマンドを実行した。 python tools/run_net.py --cfg configs/Kinetics/c2/SLOWFAST_8x8_R50.yaml DATA.PATH_TO_DATA_DIR dataset/ TEST.CHECKPOINT_FILE_PATH ./checkpoints/SLOWFAST_8x8_R50.pkl TRAIN.ENABLE False NUM_GPUS 10 TRAIN.CHECKPOINT_TYPE caffe2 TEST.CHECKPOINT_TYPE caffe2 MODEL.NUM_CLASSES 300

[INFO: logging.py: 67]: json_stats: {"split": "test_final", "top1_acc": "0.00", "top5_acc": "0.00"} のようになった。これはきちんと変化しているが、モデル構造がどうなっているかわからなかった。logが多すぎたので出力を一旦out_log.txtに保存することに。するとエラーが出ていることがわかった。

!! pred_b: (400,) does not match head.projection.bias: (300,)
!! pred_w: (400, 2304) does not match head.projection.weight: (300, 2304)
!! pred_b: (400,) does not match head.projection.bias: (300,)
!! pred_w: (400, 2304) does not match head.projection.weight: (300, 2304)
!! pred_b: (400,) does not match head.projection.bias: (300,)
!! pred_b: (400,) does not match head.projection.bias: (300,)
!! pred_w: (400, 2304) does not match head.projection.weight: (300, 2304)
!! pred_w: (400, 2304) does not match head.projection.weight: (300, 2304)
!! pred_b: (400,) does not match head.projection.bias: (300,)
!! pred_w: (400, 2304) does not match head.projection.weight: (300, 2304)
!! pred_b: (400,) does not match head.projection.bias: (300,)
!! pred_w: (400, 2304) does not match head.projection.weight: (300, 2304)
!! pred_b: (400,) does not match head.projection.bias: (300,)
!! pred_w: (400, 2304) does not match head.projection.weight: (300, 2304)
!! pred_b: (400,) does not match head.projection.bias: (300,)
!! pred_w: (400, 2304) does not match head.projection.weight: (300, 2304)

idekazuki commented 4 years ago

python tools/run_net.py --cfg configs/Kinetics/c2/SLOWFAST_8x8_R50.yaml DATA.PATH_TO_DATA_DIR dataset/ TRAIN.BATCH_SIZE 6 TRAIN.CHECKPOINT_FILE_PATH ./checkpoints/SLOWFAST_8x8_R50.pkl TRAIN.ENABLE True NUM_GPUS 10 TRAIN.CHECKPOINT_TYPE caffe2 MODEL.NUM_CLASSES 300

-- Process 1 terminated with the following error:
Traceback (most recent call last):
  File "/home/yanai-lab/ide-k/ide-k/pyenv/slowfast/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
    fn(i, *args)
  File "/home/yanai-lab/ide-k/ide-k/out_git/SlowFast-master/slowfast/utils/multiprocessing.py", line 50, in run
    func(cfg)
  File "/host/space0/ide-k/out_git/SlowFast-master/tools/train_net.py", line 276, in train
    train_loader = loader.construct_loader(cfg, "train")
  File "/home/yanai-lab/ide-k/ide-k/out_git/SlowFast-master/slowfast/datasets/loader.py", line 92, in construct_loader
    collate_fn=detection_collate if cfg.DETECTION.ENABLE else None,
  File "/home/yanai-lab/ide-k/ide-k/pyenv/slowfast/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 219, in __init__
    batch_sampler = BatchSampler(sampler, batch_size, drop_last)
  File "/home/yanai-lab/ide-k/ide-k/pyenv/slowfast/lib/python3.7/site-packages/torch/utils/data/sampler.py", line 190, in __init__
    "but got batch_size={}".format(batch_size))
ValueError: batch_size should be a positive integer value, but got batch_size=0

idekazuki commented 4 years ago

モデル構造を出力してみた。

(module): SlowFast(
    (s1): VideoModelStem(
      (pathway0_stem): ResNetBasicStem(
        (conv): Conv3d(3, 64, kernel_size=[1, 7, 7], stride=[1, 2, 2], padding=[0, 3, 3], bias=False)
        (bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (pool_layer): MaxPool3d(kernel_size=[1, 3, 3], stride=[1, 2, 2], padding=[0, 1, 1], dilation=1, ceil_mode=False)
      )
      (pathway1_stem): ResNetBasicStem(
        (conv): Conv3d(3, 8, kernel_size=[5, 7, 7], stride=[1, 2, 2], padding=[2, 3, 3], bias=False)
        (bn): BatchNorm3d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (pool_layer): MaxPool3d(kernel_size=[1, 3, 3], stride=[1, 2, 2], padding=[0, 1, 1], dilation=1, ceil_mode=False)
      )
    )
    (s1_fuse): FuseFastToSlow(
      (conv_f2s): Conv3d(8, 16, kernel_size=[7, 1, 1], stride=[4, 1, 1], padding=[3, 0, 0], bias=False)
      (bn): BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
    )
    (s2): ResStage(
      (pathway0_res0): ResBlock(
        (branch1): Conv3d(80, 256, kernel_size=(1, 1, 1), stride=[1, 1, 1], bias=False)
        (branch1_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (branch2): BottleneckTransform(
          (a): Conv3d(80, 64, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (a_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(64, 64, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(64, 256, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway0_res1): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(256, 64, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (a_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(64, 64, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(64, 256, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway0_res2): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(256, 64, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (a_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(64, 64, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(64, 256, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res0): ResBlock(
        (branch1): Conv3d(8, 32, kernel_size=(1, 1, 1), stride=[1, 1, 1], bias=False)
        (branch1_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (branch2): BottleneckTransform(
          (a): Conv3d(8, 8, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(8, 8, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(8, 32, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res1): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(32, 8, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(8, 8, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(8, 32, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res2): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(32, 8, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(8, 8, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(8, 32, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
    )
    (s2_fuse): FuseFastToSlow(
      (conv_f2s): Conv3d(32, 64, kernel_size=[7, 1, 1], stride=[4, 1, 1], padding=[3, 0, 0], bias=False)
      (bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
    )
    (pathway0_pool): MaxPool3d(kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], dilation=1, ceil_mode=False)
    (pathway1_pool): MaxPool3d(kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], dilation=1, ceil_mode=False)
    (s3): ResStage(
      (pathway0_res0): ResBlock(
        (branch1): Conv3d(320, 512, kernel_size=(1, 1, 1), stride=[1, 2, 2], bias=False)
        (branch1_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (branch2): BottleneckTransform(
          (a): Conv3d(320, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (a_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(128, 128, kernel_size=[1, 3, 3], stride=[1, 2, 2], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(128, 512, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway0_res1): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(512, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (a_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(128, 128, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(128, 512, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway0_res2): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(512, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (a_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(128, 128, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(128, 512, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway0_res3): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(512, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (a_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(128, 128, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(128, 512, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res0): ResBlock(
        (branch1): Conv3d(32, 64, kernel_size=(1, 1, 1), stride=[1, 2, 2], bias=False)
        (branch1_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (branch2): BottleneckTransform(
          (a): Conv3d(32, 16, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(16, 16, kernel_size=[1, 3, 3], stride=[1, 2, 2], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(16, 64, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res1): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(64, 16, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(16, 16, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(16, 64, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res2): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(64, 16, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(16, 16, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(16, 64, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res3): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(64, 16, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(16, 16, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(16, 64, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
    )
    (s3_fuse): FuseFastToSlow(
      (conv_f2s): Conv3d(64, 128, kernel_size=[7, 1, 1], stride=[4, 1, 1], padding=[3, 0, 0], bias=False)
      (bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
    )
    (s4): ResStage(
      (pathway0_res0): ResBlock(
        (branch1): Conv3d(640, 1024, kernel_size=(1, 1, 1), stride=[1, 2, 2], bias=False)
        (branch1_bn): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (branch2): BottleneckTransform(
          (a): Conv3d(640, 256, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(256, 256, kernel_size=[1, 3, 3], stride=[1, 2, 2], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(256, 1024, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway0_res1): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(1024, 256, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(256, 256, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(256, 1024, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway0_res2): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(1024, 256, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(256, 256, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(256, 1024, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway0_res3): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(1024, 256, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(256, 256, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(256, 1024, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway0_res4): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(1024, 256, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(256, 256, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(256, 1024, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway0_res5): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(1024, 256, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(256, 256, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(256, 1024, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res0): ResBlock(
        (branch1): Conv3d(64, 128, kernel_size=(1, 1, 1), stride=[1, 2, 2], bias=False)
        (branch1_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (branch2): BottleneckTransform(
          (a): Conv3d(64, 32, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(32, 32, kernel_size=[1, 3, 3], stride=[1, 2, 2], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(32, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res1): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(128, 32, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(32, 32, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(32, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res2): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(128, 32, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(32, 32, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(32, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res3): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(128, 32, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(32, 32, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(32, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res4): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(128, 32, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(32, 32, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(32, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res5): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(128, 32, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(32, 32, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(32, 128, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
    )
    (s4_fuse): FuseFastToSlow(
      (conv_f2s): Conv3d(128, 256, kernel_size=[7, 1, 1], stride=[4, 1, 1], padding=[3, 0, 0], bias=False)
      (bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace=True)
    )
    (s5): ResStage(
      (pathway0_res0): ResBlock(
        (branch1): Conv3d(1280, 2048, kernel_size=(1, 1, 1), stride=[1, 2, 2], bias=False)
        (branch1_bn): BatchNorm3d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (branch2): BottleneckTransform(
          (a): Conv3d(1280, 512, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(512, 512, kernel_size=[1, 3, 3], stride=[1, 2, 2], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(512, 2048, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway0_res1): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(2048, 512, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(512, 512, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(512, 2048, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway0_res2): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(2048, 512, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(512, 512, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(512, 2048, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res0): ResBlock(
        (branch1): Conv3d(128, 256, kernel_size=(1, 1, 1), stride=[1, 2, 2], bias=False)
        (branch1_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (branch2): BottleneckTransform(
          (a): Conv3d(128, 64, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(64, 64, kernel_size=[1, 3, 3], stride=[1, 2, 2], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(64, 256, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res1): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(256, 64, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(64, 64, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(64, 256, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
      (pathway1_res2): ResBlock(
        (branch2): BottleneckTransform(
          (a): Conv3d(256, 64, kernel_size=[3, 1, 1], stride=[1, 1, 1], padding=[1, 0, 0], bias=False)
          (a_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (a_relu): ReLU(inplace=True)
          (b): Conv3d(64, 64, kernel_size=[1, 3, 3], stride=[1, 1, 1], padding=[0, 1, 1], dilation=[1, 1, 1], bias=False)
          (b_bn): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (b_relu): ReLU(inplace=True)
          (c): Conv3d(64, 256, kernel_size=[1, 1, 1], stride=[1, 1, 1], padding=[0, 0, 0], bias=False)
          (c_bn): BatchNorm3d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
        (relu): ReLU(inplace=True)
      )
    )
    (head): ResNetBasicHead(
      (pathway0_avgpool): AvgPool3d(kernel_size=[8, 7, 7], stride=1, padding=0)
      (pathway1_avgpool): AvgPool3d(kernel_size=[32, 7, 7], stride=1, padding=0)
      (dropout): Dropout(p=0.5, inplace=False)
      (projection): Linear(in_features=2304, out_features=400, bias=True)
      (act): Softmax(dim=4)
    )
  )
)

idekazuki commented 4 years ago

python tools/run_net.py --cfg configs/Kinetics/SLOWFAST_8x8_R50.yaml DATA.PATH_TO_DATA_DIR ./dataset/ NUM_GPUS 2 TRAIN.BATCH_SIZE 4

idekazuki commented 4 years ago

num_features = model.module.fc.in_features↲ model.module.fc = nn.Linear(num_features,125)

上記のようにネットワーク構造を変えてみた結果、モデルの形を変えることはできたが、以下のようなデラーが出た。

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/home/yanai-lab/ide-k/ide-k/pyenv/slowfast/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
    fn(i, *args)
  File "/home/yanai-lab/ide-k/ide-k/out_git/SlowFast-master/slowfast/utils/multiprocessing.py", line 50, in run
    func(cfg)
  File "/host/space0/ide-k/out_git/SlowFast-master/tools/train_net.py", line 299, in train
    train_epoch(train_loader, model, optimizer, train_meter, cur_epoch, cfg)
  File "/host/space0/ide-k/out_git/SlowFast-master/tools/train_net.py", line 73, in train_epoch
    preds = model(inputs)
  File "/home/yanai-lab/ide-k/ide-k/pyenv/slowfast/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/yanai-lab/ide-k/ide-k/pyenv/slowfast/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 447, in forward
    output = self.module(*inputs[0], **kwargs[0])
  File "/home/yanai-lab/ide-k/ide-k/pyenv/slowfast/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/yanai-lab/ide-k/ide-k/out_git/SlowFast-master/slowfast/models/video_model_builder.py", line 377, in forward
    x = self.head(x)
  File "/home/yanai-lab/ide-k/ide-k/pyenv/slowfast/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/yanai-lab/ide-k/ide-k/out_git/SlowFast-master/slowfast/models/head_helper.py", line 209, in forward
    x = self.projection(x)
  File "/home/yanai-lab/ide-k/ide-k/pyenv/slowfast/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/yanai-lab/ide-k/ide-k/pyenv/slowfast/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 87, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/yanai-lab/ide-k/ide-k/pyenv/slowfast/lib/python3.7/site-packages/torch/nn/functional.py", line 1372, in linear
    output = input.matmul(weight.t())
RuntimeError: Expected object of device type cuda but got device type cpu for argument #2 'mat2' in call to _th_mm

idekazuki / diary

SlowFast をfine-tuning #79