search

beacon-biosignals / Lighthouse.jl

Performance evaluation tools for multiclass, multirater classification models
MIT License
18 stars 2 forks source link

`evaluation_metrics_plot` fails when IRA fields are in the input `EvaluationRow` for multiclass result #75

Closed alexmchan closed 2 years ago

alexmchan commented 2 years ago

Currently, if votes is not provided to Lighthouse.evaluation_metrics_row, it outputs a row with per_class_IRA_kappas and multiclass_IRA_kappas fields defined as missing. If this is a multiclass result, then passing in such a row to evaluation_metrics_plot fails because it doesn't expect these fields to exist when not dealing with multi-rater metrics.

For example, for a 5 class EvaluationRow, it throws:

ERROR: DimensionMismatch("arrays could not be broadcast to a common size; got a dimension with lengths 12 and 8")
hannahilea commented 2 years ago

oh geez. this will be an easy fix---I can make it for you asap, @alexmchan, unless you're already on it?

hannahilea commented 2 years ago

yet another reason that it will be exciting to do #71---plots shouldn't ask for more than they truly need/expect to plot. none of this evaluation_metrics_plot behemoth nonsense.

alexmchan commented 2 years ago

I'm already mostly through this one. Thanks for the offer though!