When to filter by prediction and class?

[gumityolcu]

Argument to filter in metrics: It makes conceptual sense to do the filtering inside the metric, is also better for a user that has precomputed explanations.

Argument to filter in benchmarks, before updating metric: Explanation is expensive. It's just a waste of time to compute unneeded explanations in the benchmark

[dilyabareeva]

Why not both?

We can leave the filtering functionality in the metric

And do pre-filtering in the benchmark if the argument is filter_by_prediction/class = True, to not calculate unnecessary explanations

In downloadable benchmarks I did the pre-filtering step