Issues about the test video list and the evaluation result

[Pilhyeon]

Hello. First of all, thanks for your good reproducing of STPN. I have two issues. One is about the test video list, and the other is about the evaluation result.

1. In the THUMOS'14 test split, there are 213 videos. After removing two videos(270, 1496), there will be 211 videos. However there are 210 videos in your test split file. I found that the difference is existence of number "1292". Is there any reason for not including that video? p.s. When I included that video, the performance decreased.

2. I think I perfectly followed your instruction. But there is slight difference between my evaluation result and yours. My result is like below: | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 | mAP | | 0.5220 | 0.4356 | 0.3331 | 0.2449 | 0.1675 | 0.0987 | 00474 | 0.0126 | 0.0009 | 0.2070 | I used your code and ckpt files. Is there any factor of making differences? (randomness, different hyper-parameter, etc.)

Thanks for your good work, again.

[bellos1203]

Hi, First, thank you for your interest.

1. Well, many papers exclude the video "1292", because it doesn't have any ground truth action annotation. The video seems like containing "Golf Swing", actually it doesn't. The video belongs to the "Ambiguous" annotations.

2. I'm not sure, but maybe there could be little difference when preprocessing, such as extracting optical flow before/after rescaling the video, etc. It seems the numbers are similar to the paper, so I think it's not a big deal.

Thanks :)

[ahmdtaha]

I am trying to replicate the results in this repos but my results lag as follows

| 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 | mAP | 0.499| 0.416| 0.324|0.239|0.166| 0.100|0.043|0.012|0.001|0.2004153548431421

I use

• ffmpeg to subsample 10fps
• dense_flow (cv::cuda::OpticalFlowDual_TVL1::create()) with param bound=20

did you use these tools or something else?

• I don't think subsampling 1 frame every 3 frames is accurate because some videos have less than 30 fps (e.g., 24 fps).
• I preserve the aspect ratio when scaling the frames

Any idea what could be wrong?