whats the meaning of num frames, sampling rate and slowfast alpha?

[Leozyc-waseda]

In yaml file, i found a lot of paramaters, but i cant understand the meaning of it very well.

For example, i want to change the input frame rate, which paramaters i should to change, NUM_FRAMES? whats meaning of NUM_FRAMES?? If have 75frame, then average choose 32 frame?

DATA: NUM_FRAMES: 32 SAMPLING_RATE: 3 TRAIN_JITTER_SCALES: [256, 320] TRAIN_CROP_SIZE: 224 TEST_CROP_SIZE: 256 INPUT_CHANNEL_NUM: [3, 3] SLOWFAST: ALPHA: 4 BETA_INV: 8 FUSION_CONV_CHANNEL_RATIO: 2 FUSION_KERNEL_SZ: 7

[alpargun]

• NUM_FRAMES: number of frames to be processed for each iteration of training or inference. (e.g. for NUM_FRAMES:32, you pass 32 frames (images) at the same time for the video network to process).
• SAMPLING RATE: the sampling rate that the video is sampled. Depends on how you would like to process your video dataset to sample frames (images). The preprocessors use this information to prepare the training samples.
• TRAIN_JITTER_SCALES, TRAIN_CROP_SIZE, TEST_CROP_SIZE: data augmentation to apply transformations to your inputs. E.g. for self-supervised learning, heavy data augmentations are used to obtain two different views of an input to compare with each other, and the other queries from the dictionary. You can use the default setting for TRAIN_JITTER_SCALES but TRAIN_CROP_SIZE, TEST_CROP_SIZE also depends on your hardware as there is a trade-off between performance and accuracy. If you can afford it, 224 and 256 are default, respectively.

For the rest, which are SlowFast algorithm hyperparameters, please refer to defaults.py and the SlowFast paper for the theory. But in short: SLOWFAST hyperparameters:

• ALPHA: frame rate reduction ratio between Slow and Fast pathways
• BETA_INV: inverse of the channel reduction ratio between Slow and Fast pathways.
• INPUT_CHANNEL_NUM: leave it [3,3] if you are using RGB input
• FUSION_KERNEL_SZ: Kernel dimension used for fusing information from Fast to Slow pathways
• FUSION_CONV_CHANNEL_RATIO: Ratio of channel dimensions between the Slow and Fast pathways As these are very specific, I would recommend starting with the default values or using the values in the configs/ directory.