amazon-science / long-short-term-transformer

[NeurIPS 2021 Spotlight] Official implementation of Long Short-Term Transformer for Online Action Detection
Apache License 2.0
125 stars 19 forks source link

some questions about rgb feature and flow feature #24

Closed SugarMuzi closed 1 year ago

SugarMuzi commented 1 year ago

I had extracted flow using denseflow with the command:denseflow videolist_all.txt -b=20 -a=tvl1 -s=2 -v. It work well. The flow had been saved by flowx{:%d5}.jpg and flowy{:%d5}.jpg file format. e.g:there has 6 rgb frames,index from 0 to 5.so the flow data have 10 flow frames,index from flow_x_00000.jpg to from_x_00004.jpg and flow_y_00000.jpg to flow_y_00004.jpg respectively,so the number of flow frames is 10. And then i use tsn_feature_extraction.py to extract rgb feature and flow feature,i get the rgb feature shape is [6,2048] and flow feature is [10,2048],but, i think the flow feature shape should be [5,2048].So,is there any problem here? My data pipeline is data_pipeline = [ dict( type='CollectFrames', clip_len=args.clip_len, frame_interval=2,

num_clips=25,

            test_mode=True),
        dict(type='RawFrameDecode'),
        dict(type='Resize', scale=(-1, 256)),
        dict(type='TenCrop', crop_size=224),
        dict(type='Normalize', **args.img_norm_cfg),
        dict(type='FormatShape', input_format='NCHW_Flow'),
        dict(type='Collect', keys=['imgs'], meta_keys=[]),
        dict(type='ToTensor', keys=['imgs'])
prozzzzzz commented 1 year ago

@SugarMuzi Hi, Has the problem been solved? I am also extracting. Can you share the detailed extraction process?

Chenhongchang commented 8 months ago

@SugarMuzi @SugarMuzi Hi, Has the problem been solved?

SugarMuzi commented 6 months ago

@SugarMuzi @SugarMuzi Hi, Has the problem been solved?

hi~ I think I have found the problem. In other issues of this GitHub repository, someone asked about the handling after extracting optical flow features using bn-inception. The author replied that they used average pooling. Therefore, after extracting optical flow features using tsn_feature_extraction, you just need to use average pooling for dimensionality reduction.

SugarMuzi commented 6 months ago

@SugarMuzi Hi, Has the problem been solved? I am also extracting. Can you share the detailed extraction process?

hi~ The above answer is my solution.

Chenhongchang commented 5 months ago

@SugarMuzi @SugarMuzi Hi, Has the problem been solved?

hi~ I think I have found the problem. In other issues of this GitHub repository, someone asked about the handling after extracting optical flow features using bn-inception. The author replied that they used average pooling. Therefore, after extracting optical flow features using tsn_feature_extraction, you just need to use average pooling for dimensionality reduction.

Thank you for your detailed response!