TL;DR - It’s time to stop reporting just MAE.
Reason: Look at standard deviations associated with MAE of SOTA approaches here . MAE is not a reliable summary measure of performance across the entire count range.
What to report: MAE and deviation at different count ranges, as shown here . This notebook can be extended with results from new networks.
Paper Title: Wisdom of (Binned) Crowds: A Bayesian Stratification Paradigm for Crowd Counting
Conference: ACMMM 2021
repo project page paper
TL;DR - It’s time to stop reporting just MAE. Reason: Look at standard deviations associated with MAE of SOTA approaches here . MAE is not a reliable summary measure of performance across the entire count range.
What to report: MAE and deviation at different count ranges, as shown here . This notebook can be extended with results from new networks.