search

athaddius / STIRMetrics

Metric Evaluation for Models on STIR
MIT License
4 stars 1 forks source link

About the competition evaluation indicators and the rules of the competition #4

Closed SzuPc closed 2 months ago

SzuPc commented 3 months ago

Thank you for answering my last question. I noticed that the accuracy evaluation index in the official website of the competition comes from the relevant code of TAP-Vid's evaluation_datasets.py. However, the STIRMetrics code does not provide the corresponding evaluation method, but only provides the calculation of epe and chamer. In addition, TAP mainly calculates points within threshold. I want to know when there will be specific code and the accuracy index of RAFT, CSRT and other models instead of epe value. In addition, I also want to know when the deadline for submitting the entire competition code is?Thank you for your reply. I hope you can contact me at any time if you have any questions！

athaddius commented 2 months ago

Hi, of course! We are planning to have the deadline for submission in early September. As for accuracy code using a TAP-based threshold metric, expect it to be available by next week.