Echo0125 / MAT-Memory-and-Anticipation-Transformer

[ICCV 2023] Official implementation of Memory-and-Anticipation Transformer for Online Action Understanding
Apache License 2.0
44 stars 3 forks source link

The precision with batch inference is not as good as when trained #14

Closed mugenggeng closed 3 months ago

mugenggeng commented 5 months ago

Sorry to bother you, I recently had a problem with the batch test model: the accuracy obtained with the batch test was not as good as the test accuracy obtained during training. Do you know why?

Echo0125 commented 5 months ago

Referring to line 53 in the dataset perframe_data_layer.py, the data is sparsely sampled during training, including train data and val data, so the performance will be different from the final test.

mugenggeng commented 5 months ago

Thank you very much for your answer. Could you please provide us with the config file of HDD data set? @Echo0125

Echo0125 commented 5 months ago

You can use the model parameters in thumos' config as hyperparameters. In addition, you can refer to the relevant hyperparameters mentioned in the supplementary material for modification.

mugenggeng commented 5 months ago

Thanks for your help, I also find that there seems to be no way to read HDD datasets in perframe_data_layers.py @Echo0125

Echo0125 commented 5 months ago

You can prepare the dataset according to this link.

Prism-hua commented 5 months ago

Hello @Echo0125 , I have downloaded the HDD dataset and found the folder of sensor features, but I do not know what kind of input type the sensor features belong to, rgb or optical flow? Because I have found only one folder, in contrast with the data structure of TVSeries mentioned in the README, which contains two separate folders for rgb inputs and optical flow inputs respectively.

Here is all the files I get after applied for the HDD dataset: image

mugenggeng commented 5 months ago

image I am very sorry to bother you again, but I would like to know how the data in Table 8 was obtained. Is there a script that can be run directly?

Echo0125 commented 3 months ago

Sorry for the late reply. For parameters other than the Model, we referred to the previous papers. As for the Model, you can manually write a for-loop script to perform the necessary calculations.